2024-05-08T08:14:16.771 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T08:14:16.771 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_teuthology@teuthology --description rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} --archive /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986 --name teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi -- /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/orig.config.yaml 2024-05-08T08:14:16.804 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T08:14:16.880 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986 branch: squid description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} email: ceph-qa@ceph.com first_in_suite: false job_id: '7695986' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: rbd qos iops burst: 100 rbd qos iops limit: 50 rbd qos schedule tick min: 100 global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: zstd bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 5498 sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc sleep_before_teardown: 0 subset: 27/128 suite: rbd suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc targets: smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNejBCF/DuKPLpP+eWAWmAkE0gfzD0e90sXSaPzmuiG8JwsuPdxq6g0V8DXPTU4/E1QaKQfHC8Pr2oLlvTh8WjY= smithi149.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ45Y2WReKifkeHkMYrHECOm6KXf+BP7sUUehrn01mJX4P7Htv8/19aiyf+WmSH2EOVLQ60XiKCOWNuPIQfmr3o= tasks: - install: null - ceph: null - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-07_21:16:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T08:14:16.880 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa; will attempt to use it 2024-05-08T08:14:16.881 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa/tasks 2024-05-08T08:14:16.881 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T08:14:16.883 INFO:teuthology.task.internal:Checking packages... 2024-05-08T08:14:16.906 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc' 2024-05-08T08:14:16.907 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T08:14:16.907 INFO:teuthology.packaging:ref: None 2024-05-08T08:14:16.907 INFO:teuthology.packaging:tag: None 2024-05-08T08:14:16.907 INFO:teuthology.packaging:branch: squid 2024-05-08T08:14:16.907 INFO:teuthology.packaging:sha1: 96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:14:16.907 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-05-08T08:14:17.109 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2515.g96805dcf 2024-05-08T08:14:17.111 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T08:14:17.118 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T08:14:17.118 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T08:14:17.138 INFO:teuthology.task.internal:Saving configuration 2024-05-08T08:14:17.151 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T08:14:17.157 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T08:14:17.183 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 08:09:17.610421', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNejBCF/DuKPLpP+eWAWmAkE0gfzD0e90sXSaPzmuiG8JwsuPdxq6g0V8DXPTU4/E1QaKQfHC8Pr2oLlvTh8WjY='} 2024-05-08T08:14:17.206 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi149.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 08:09:17.608488', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ45Y2WReKifkeHkMYrHECOm6KXf+BP7sUUehrn01mJX4P7Htv8/19aiyf+WmSH2EOVLQ60XiKCOWNuPIQfmr3o='} 2024-05-08T08:14:17.207 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T08:14:17.213 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-08T08:14:17.213 INFO:teuthology.task.internal:roles: ubuntu@smithi149.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-08T08:14:17.213 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T08:14:17.356 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb6ac169e50>, signals=[15]) 2024-05-08T08:14:17.357 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T08:14:17.363 INFO:teuthology.task.internal:Opening connections... 2024-05-08T08:14:17.364 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:14:17.365 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:14:17.441 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:14:17.442 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:14:17.516 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T08:14:17.523 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-05-08T08:14:17.542 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-05-08T08:14:17.542 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-05-08T08:14:17.600 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="CentOS Stream" 2024-05-08T08:14:17.600 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="9" 2024-05-08T08:14:17.600 INFO:teuthology.orchestra.run.smithi031.stdout:ID="centos" 2024-05-08T08:14:17.600 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE="rhel fedora" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="9" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:PLATFORM_ID="platform:el9" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:ANSI_COLOR="0;31" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:LOGO="fedora-logo-icon" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://centos.org/" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T08:14:17.601 INFO:teuthology.orchestra.run.smithi031.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T08:14:17.602 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-05-08T08:14:17.642 DEBUG:teuthology.orchestra.run.smithi149:> uname -m 2024-05-08T08:14:17.660 INFO:teuthology.orchestra.run.smithi149.stdout:x86_64 2024-05-08T08:14:17.660 DEBUG:teuthology.orchestra.run.smithi149:> cat /etc/os-release 2024-05-08T08:14:17.716 INFO:teuthology.orchestra.run.smithi149.stdout:NAME="CentOS Stream" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION="9" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:ID="centos" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:ID_LIKE="rhel fedora" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:VERSION_ID="9" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:PLATFORM_ID="platform:el9" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:ANSI_COLOR="0;31" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:LOGO="fedora-logo-icon" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:HOME_URL="https://centos.org/" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T08:14:17.717 INFO:teuthology.orchestra.run.smithi149.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T08:14:17.718 INFO:teuthology.lock.ops:Updating smithi149.front.sepia.ceph.com on lock server 2024-05-08T08:14:17.741 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T08:14:17.750 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T08:14:17.755 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T08:14:17.756 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-05-08T08:14:17.758 DEBUG:teuthology.orchestra.run.smithi149:> test '!' -e /home/ubuntu/cephtest 2024-05-08T08:14:17.774 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T08:14:17.780 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T08:14:17.781 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-05-08T08:14:17.816 DEBUG:teuthology.orchestra.run.smithi149:> test -z $(ls -A /var/lib/ceph) 2024-05-08T08:14:17.845 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T08:14:17.893 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T08:14:17.906 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T08:14:17.907 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T08:14:17.907 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T08:14:17.907 DEBUG:teuthology.orchestra.run.smithi149:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T08:14:17.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:14:17.923 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-05-08T08:14:17.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:14:17.924 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-05-08T08:14:17.980 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-437.el9.x86_64 2024-05-08T08:14:17.980 INFO:teuthology.task.kernel:Running kernel on smithi149: 5.14.0-437.el9.x86_64 2024-05-08T08:14:17.980 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum install -y kernel 2024-05-08T08:14:17.982 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-437.el9.x86_64 2024-05-08T08:14:17.983 INFO:teuthology.task.kernel:Running kernel on smithi031: 5.14.0-437.el9.x86_64 2024-05-08T08:14:17.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum install -y kernel 2024-05-08T08:14:20.218 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-08T08:14:20.786 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.1 MB 00:01 2024-05-08T08:14:36.977 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:15 2024-05-08T08:14:37.834 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-05-08T08:14:41.269 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-08T08:14:42.992 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.2 MB 00:01 2024-05-08T08:14:42.994 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 16 kB 00:00 2024-05-08T08:14:46.026 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 9.8 kB/s | 16 kB 00:01 2024-05-08T08:14:48.157 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 21 MB 00:05 2024-05-08T08:14:49.822 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 5.8 MB/s | 21 MB 00:03 2024-05-08T08:14:53.511 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-08T08:14:55.135 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-05-08T08:14:55.601 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T08:14:55.601 INFO:teuthology.orchestra.run.smithi031.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T08:14:55.684 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:14:55.695 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:14:55.696 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-05-08T08:14:55.697 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:14:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 78 M 2024-05-08T08:14:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 123 M 2024-05-08T08:14:55.697 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:14:56.946 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 409 kB/s | 484 kB 00:01 2024-05-08T08:14:57.142 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T08:14:57.142 INFO:teuthology.orchestra.run.smithi149.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T08:14:57.236 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:14:57.253 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:14:57.253 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-05-08T08:14:57.253 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:14:57.253 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:14:57.253 INFO:teuthology.orchestra.run.smithi149.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T08:14:57.254 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:14:57.254 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T08:14:57.254 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T08:14:57.254 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T08:14:57.254 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:14:57.254 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:14:57.255 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:14:57.255 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-05-08T08:14:57.255 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:14:57.255 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 78 M 2024-05-08T08:14:57.255 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 123 M 2024-05-08T08:14:57.256 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:14:58.045 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.1 MB/s | 484 kB 00:00 2024-05-08T08:14:58.646 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 5.3 MB/s | 15 MB 00:02 2024-05-08T08:14:59.520 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 8.1 MB/s | 15 MB 00:01 2024-05-08T08:14:59.862 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 15 MB/s | 28 MB 00:01 2024-05-08T08:15:00.287 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 13 MB/s | 34 MB 00:02 2024-05-08T08:15:00.288 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:15:00.288 INFO:teuthology.orchestra.run.smithi149.stdout:Total 26 MB/s | 78 MB 00:03 2024-05-08T08:15:00.688 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 6.9 MB/s | 34 MB 00:04 2024-05-08T08:15:00.969 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:15:01.337 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:15:01.337 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:15:01.346 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 6.4 MB/s | 28 MB 00:04 2024-05-08T08:15:01.346 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:15:01.346 INFO:teuthology.orchestra.run.smithi031.stdout:Total 14 MB/s | 78 MB 00:05 2024-05-08T08:15:02.058 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:15:02.260 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:15:02.260 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:15:02.492 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:15:02.492 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:15:03.447 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:15:03.447 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:15:04.694 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:15:05.348 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:15:05.462 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:15:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:15:06.663 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:15:06.745 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:15:08.140 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:15:08.288 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:15:09.562 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:15:09.694 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:15:13.018 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:15:13.066 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:14.776 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:15:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:17.809 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:19.910 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:47.476 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:47.515 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:47.888 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:47.888 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:15:47.888 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:15:47.888 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:15:48.697 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:48.697 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:15:48.697 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:15:48.697 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T08:15:48.698 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T08:15:48.698 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T08:15:48.698 INFO:teuthology.orchestra.run.smithi149.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T08:15:48.698 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:15:48.698 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:15:49.055 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:15:49.134 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:15:49.135 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:15:49.135 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T08:15:49.135 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T08:15:49.135 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T08:15:49.135 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-08T08:15:49.136 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi149.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T08:15:49.136 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:15:49.186 INFO:teuthology.orchestra.run.smithi149.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:15:49.186 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:15:49.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo rpm -qi grub2-tools 2024-05-08T08:15:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:Name : grub2-tools 2024-05-08T08:15:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:Epoch : 1 2024-05-08T08:15:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:Version : 2.06 2024-05-08T08:15:49.224 INFO:teuthology.orchestra.run.smithi149.stdout:Release : 68.el9 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Architecture: x86_64 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Group : Unspecified 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Size : 8264254 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:License : GPLv3+ 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Packager : builder@centos.org 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Vendor : CentOS 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T08:15:49.225 INFO:teuthology.orchestra.run.smithi149.stdout:Summary : Support tools for GRUB. 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:Description : 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:hardware devices. 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:15:49.226 INFO:teuthology.orchestra.run.smithi149.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T08:15:49.227 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T08:15:49.227 INFO:teuthology.task.kernel:Updating grub on smithi149 to boot 5.14.0-444.el9.x86_64 2024-05-08T08:15:49.227 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T08:15:49.836 INFO:teuthology.orchestra.run.smithi149.stderr:Generating grub configuration file ... 2024-05-08T08:15:50.756 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:50.796 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:51.084 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:51.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T08:15:51.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T08:15:51.084 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T08:15:51.213 INFO:teuthology.orchestra.run.smithi149.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T08:15:51.245 INFO:teuthology.orchestra.run.smithi149.stderr:done 2024-05-08T08:15:51.247 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-05-08T08:15:51.263 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.JH1Ep26LN4 2024-05-08T08:15:51.263 DEBUG:teuthology.orchestra.run.smithi149:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.JH1Ep26LN4 2024-05-08T08:15:51.340 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.JH1Ep26LN4 2024-05-08T08:15:51.473 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.JH1Ep26LN4 is 6KB 2024-05-08T08:15:51.522 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.JH1Ep26LN4 2024-05-08T08:15:51.536 DEBUG:teuthology.orchestra.run.smithi149:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T08:15:51.597 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T08:15:51.636 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T08:15:51.636 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T08:15:51.636 INFO:teuthology.orchestra.run.smithi149.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T08:15:51.637 DEBUG:teuthology.orchestra.run.smithi149:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T08:15:51.777 DEBUG:teuthology.orchestra.run.smithi149:> sudo shutdown -r now 2024-05-08T08:15:51.890 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:15:51.891 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:15:52.259 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:15:52.337 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:15:52.338 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:15:52.338 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T08:15:52.338 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T08:15:52.338 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T08:15:52.338 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T08:15:52.338 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi031.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T08:15:52.338 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T08:15:52.389 INFO:teuthology.orchestra.run.smithi031.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T08:15:52.389 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T08:15:52.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo rpm -qi grub2-tools 2024-05-08T08:15:52.420 INFO:teuthology.orchestra.run.smithi031.stdout:Name : grub2-tools 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Epoch : 1 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Version : 2.06 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Release : 68.el9 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Architecture: x86_64 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Group : Unspecified 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Size : 8264254 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:License : GPLv3+ 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Packager : builder@centos.org 2024-05-08T08:15:52.421 INFO:teuthology.orchestra.run.smithi031.stdout:Vendor : CentOS 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:Summary : Support tools for GRUB. 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:Description : 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:hardware devices. 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:15:52.422 INFO:teuthology.orchestra.run.smithi031.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T08:15:52.423 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T08:15:52.423 INFO:teuthology.task.kernel:Updating grub on smithi031 to boot 5.14.0-444.el9.x86_64 2024-05-08T08:15:52.423 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T08:15:53.027 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-05-08T08:15:54.407 INFO:teuthology.orchestra.run.smithi031.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T08:15:54.438 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-05-08T08:15:54.440 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-05-08T08:15:54.455 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.xQrgRrA9GP 2024-05-08T08:15:54.456 DEBUG:teuthology.orchestra.run.smithi031:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.xQrgRrA9GP 2024-05-08T08:15:54.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.xQrgRrA9GP 2024-05-08T08:15:54.661 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.xQrgRrA9GP is 6KB 2024-05-08T08:15:54.681 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.xQrgRrA9GP 2024-05-08T08:15:54.696 DEBUG:teuthology.orchestra.run.smithi031:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T08:15:54.758 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T08:15:54.758 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T08:15:54.758 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T08:15:54.759 INFO:teuthology.orchestra.run.smithi031.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T08:15:54.760 DEBUG:teuthology.orchestra.run.smithi031:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T08:15:54.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo shutdown -r now 2024-05-08T08:16:21.804 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-08T08:16:21.805 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-08T08:16:21.807 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:16:24.918 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T08:16:24.918 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-05-08T08:16:24.919 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:16:40.222 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-05-08T08:16:43.294 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-05-08T08:16:49.228 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-08T08:16:49.229 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:16:52.286 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-05-08T08:16:52.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-05-08T08:16:52.295 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:17:04.297 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-08T08:17:04.298 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:17:10.686 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.31 2024-05-08T08:17:22.698 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-05-08T08:17:22.700 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:17:22.722 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.149 2024-05-08T08:17:23.070 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-05-08T08:17:23.541 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-05-08T08:17:23.541 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-08T08:17:23.541 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-05-08T08:17:23.556 INFO:teuthology.orchestra.run.smithi031.stdout:5.14.0-444.el9.x86_64 2024-05-08T08:17:23.557 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T08:17:23.557 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T08:17:23.557 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T08:17:24.557 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T08:17:24.558 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T08:17:24.644 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-05-08T08:17:24.696 DEBUG:teuthology.parallel:result is None 2024-05-08T08:17:37.723 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-08T08:17:37.724 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:17:38.140 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-08T08:17:38.523 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-08T08:17:38.523 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-08T08:17:38.524 DEBUG:teuthology.orchestra.run.smithi149:> uname -r 2024-05-08T08:17:38.538 INFO:teuthology.orchestra.run.smithi149.stdout:5.14.0-444.el9.x86_64 2024-05-08T08:17:38.538 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T08:17:38.539 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T08:17:38.539 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T08:17:39.540 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T08:17:39.540 DEBUG:teuthology.orchestra.run.smithi149:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T08:17:39.617 INFO:teuthology.orchestra.run.smithi149.stdout:ttyS1 2024-05-08T08:17:39.648 DEBUG:teuthology.parallel:result is None 2024-05-08T08:17:39.648 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T08:17:39.656 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T08:17:39.656 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T08:17:39.659 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T08:17:39.677 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T08:17:39.683 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T08:17:39.691 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T08:17:39.692 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T08:17:39.719 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T08:17:39.753 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T08:17:39.760 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T08:17:39.760 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T08:17:39.790 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T08:17:39.820 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:17:39.832 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:17:39.832 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:17:39.842 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T08:17:39.844 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T08:17:39.850 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T08:17:39.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T08:17:39.877 DEBUG:teuthology.orchestra.run.smithi149:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T08:17:39.907 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T08:17:39.915 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T08:17:39.915 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T08:17:39.947 DEBUG:teuthology.orchestra.run.smithi149:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T08:17:39.963 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:17:40.050 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:17:40.097 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:17:40.183 DEBUG:teuthology.orchestra.run.smithi031:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:17:40.226 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:17:40.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T08:17:40.293 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:17:40.333 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T08:17:40.383 DEBUG:teuthology.orchestra.run.smithi149:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:17:40.460 DEBUG:teuthology.orchestra.run.smithi149:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T08:17:40.499 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:17:40.499 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T08:17:40.560 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-05-08T08:17:40.563 DEBUG:teuthology.orchestra.run.smithi149:> sudo service rsyslog restart 2024-05-08T08:17:40.613 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:17:40.655 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:17:41.119 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T08:17:41.127 INFO:teuthology.task.internal:Starting timer... 2024-05-08T08:17:41.128 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T08:17:41.138 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T08:17:41.147 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T08:17:41.147 DEBUG:teuthology.orchestra.run.smithi031:> sudo service auditd rotate 2024-05-08T08:17:41.214 INFO:teuthology.orchestra.run.smithi031.stdout:Rotating logs: 2024-05-08T08:17:41.216 DEBUG:teuthology.orchestra.run.smithi149:> sudo service auditd rotate 2024-05-08T08:17:41.337 INFO:teuthology.orchestra.run.smithi149.stdout:Rotating logs: 2024-05-08T08:17:41.339 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T08:17:41.340 DEBUG:teuthology.orchestra.run.smithi031:> /usr/sbin/getenforce 2024-05-08T08:17:41.367 INFO:teuthology.orchestra.run.smithi031.stdout:Permissive 2024-05-08T08:17:41.368 DEBUG:teuthology.orchestra.run.smithi149:> /usr/sbin/getenforce 2024-05-08T08:17:41.391 INFO:teuthology.orchestra.run.smithi149.stdout:Permissive 2024-05-08T08:17:41.391 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi031.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi149.front.sepia.ceph.com': 'permissive'} 2024-05-08T08:17:41.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:17:41.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:17:41.438 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:17:41.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:17:41.463 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T08:17:41.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo /usr/sbin/setenforce permissive 2024-05-08T08:17:41.511 DEBUG:teuthology.orchestra.run.smithi149:> sudo /usr/sbin/setenforce permissive 2024-05-08T08:17:41.543 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T08:17:41.553 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T08:17:41.559 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T08:17:41.687 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T08:17:41.702 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T08:17:41.703 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi031.front.sepia.ceph.com,smithi149.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T08:21:43.378 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi031.front.sepia.ceph.com'), Remote(name='ubuntu@smithi149.front.sepia.ceph.com')] 2024-05-08T08:21:43.379 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-05-08T08:21:43.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:21:43.458 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-05-08T08:21:43.539 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-05-08T08:21:43.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-08T08:21:43.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi149.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T08:21:43.615 DEBUG:teuthology.orchestra.run.smithi149:> true 2024-05-08T08:21:43.694 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi149.front.sepia.ceph.com' 2024-05-08T08:21:43.694 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T08:21:43.705 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T08:21:43.706 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T08:21:43.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:21:43.709 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T08:21:43.709 DEBUG:teuthology.orchestra.run.smithi149:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:21:43.739 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T08:21:43.754 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T08:21:43.764 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T08:21:43.778 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T08:21:43.781 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: ntpd: command not found 2024-05-08T08:21:43.793 INFO:teuthology.orchestra.run.smithi031.stdout:506 Cannot talk to daemon 2024-05-08T08:21:43.803 INFO:teuthology.orchestra.run.smithi149.stderr:sudo: ntpd: command not found 2024-05-08T08:21:43.808 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T08:21:43.814 INFO:teuthology.orchestra.run.smithi149.stdout:506 Cannot talk to daemon 2024-05-08T08:21:43.823 INFO:teuthology.orchestra.run.smithi031.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T08:21:43.829 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T08:21:43.843 INFO:teuthology.orchestra.run.smithi149.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T08:21:43.880 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:21:43.884 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:21:43.884 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-05-08T08:21:43.884 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.884 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.884 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.884 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.898 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:21:43.902 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:21:43.902 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-08T08:21:43.902 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.902 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.902 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.902 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:21:43.903 INFO:teuthology.run_tasks:Running task install... 2024-05-08T08:21:43.912 DEBUG:teuthology.task.install:project ceph 2024-05-08T08:21:43.912 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-08T08:21:43.912 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'} 2024-05-08T08:21:43.912 INFO:teuthology.task.install:Using flavor: default 2024-05-08T08:21:43.919 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T08:21:43.919 INFO:teuthology.task.install:extra packages: [] 2024-05-08T08:21:43.919 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-08T08:21:43.919 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:21:43.922 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'tag': None, 'wait_for_package': False} 2024-05-08T08:21:43.922 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:21:44.066 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-08T08:21:44.066 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-08T08:21:44.142 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/ 2024-05-08T08:21:44.142 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2515.g96805dcf 2024-05-08T08:21:44.287 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T08:21:44.288 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:21:44.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T08:21:44.316 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T08:21:44.317 DEBUG:teuthology.orchestra.run.smithi031:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T08:21:44.363 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T08:21:44.363 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:21:44.363 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T08:21:44.384 DEBUG:teuthology.orchestra.run.smithi031:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T08:21:44.392 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T08:21:44.392 DEBUG:teuthology.orchestra.run.smithi149:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T08:21:44.459 DEBUG:teuthology.orchestra.run.smithi149:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T08:21:44.461 DEBUG:teuthology.orchestra.run.smithi031:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T08:21:44.526 INFO:teuthology.orchestra.run.smithi031.stdout:check_obsoletes = 1 2024-05-08T08:21:44.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum clean all 2024-05-08T08:21:44.535 DEBUG:teuthology.orchestra.run.smithi149:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T08:21:44.600 INFO:teuthology.orchestra.run.smithi149.stdout:check_obsoletes = 1 2024-05-08T08:21:44.601 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum clean all 2024-05-08T08:21:44.763 INFO:teuthology.orchestra.run.smithi031.stdout:54 files removed 2024-05-08T08:21:44.791 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-radosgw 2024-05-08T08:21:44.833 INFO:teuthology.orchestra.run.smithi149.stdout:54 files removed 2024-05-08T08:21:44.860 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-radosgw 2024-05-08T08:21:45.723 INFO:teuthology.orchestra.run.smithi031.stdout:ceph packages for x86_64 129 kB/s | 83 kB 00:00 2024-05-08T08:21:45.799 INFO:teuthology.orchestra.run.smithi149.stdout:ceph packages for x86_64 126 kB/s | 83 kB 00:00 2024-05-08T08:21:46.415 INFO:teuthology.orchestra.run.smithi031.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-05-08T08:21:46.491 INFO:teuthology.orchestra.run.smithi149.stdout:ceph noarch packages 18 kB/s | 11 kB 00:00 2024-05-08T08:21:47.041 INFO:teuthology.orchestra.run.smithi031.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-05-08T08:21:47.100 INFO:teuthology.orchestra.run.smithi149.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-05-08T08:21:47.558 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.1 MB 00:00 2024-05-08T08:21:47.675 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-08T08:21:50.317 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-05-08T08:21:51.351 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-05-08T08:21:54.884 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - CRB 7.9 MB/s | 6.2 MB 00:00 2024-05-08T08:21:56.419 INFO:teuthology.orchestra.run.smithi031.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-05-08T08:21:57.589 INFO:teuthology.orchestra.run.smithi031.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:01 2024-05-08T08:21:58.726 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - CRB 1.6 MB/s | 6.2 MB 00:03 2024-05-08T08:22:00.410 INFO:teuthology.orchestra.run.smithi149.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-05-08T08:22:01.403 INFO:teuthology.orchestra.run.smithi149.stdout:Extra Packages for Enterprise Linux 25 MB/s | 21 MB 00:00 2024-05-08T08:22:02.970 INFO:teuthology.orchestra.run.smithi031.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-05-08T08:22:05.433 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:22:05.441 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrading: 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-08T08:22:05.442 INFO:teuthology.orchestra.run.smithi031.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-08T08:22:05.443 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout:Installing weak dependencies: 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout:Install 30 Packages 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout:Upgrade 2 Packages 2024-05-08T08:22:05.444 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 60 M 2024-05-08T08:22:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:22:06.704 INFO:teuthology.orchestra.run.smithi149.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-05-08T08:22:07.538 INFO:teuthology.orchestra.run.smithi031.stdout:(1/32): ceph-base-19.0.0-2515.g96805dcf.el9.x86 3.2 MB/s | 5.4 MB 00:01 2024-05-08T08:22:07.606 INFO:teuthology.orchestra.run.smithi031.stdout:(2/32): ceph-selinux-19.0.0-2515.g96805dcf.el9. 379 kB/s | 25 kB 00:00 2024-05-08T08:22:07.789 INFO:teuthology.orchestra.run.smithi031.stdout:(3/32): ceph-radosgw-19.0.0-2515.g96805dcf.el9. 5.4 MB/s | 10 MB 00:01 2024-05-08T08:22:07.822 INFO:teuthology.orchestra.run.smithi031.stdout:(4/32): libcephfs2-19.0.0-2515.g96805dcf.el9.x8 3.4 MB/s | 746 kB 00:00 2024-05-08T08:22:07.864 INFO:teuthology.orchestra.run.smithi031.stdout:(5/32): libradosstriper1-19.0.0-2515.g96805dcf. 6.5 MB/s | 500 kB 00:00 2024-05-08T08:22:07.922 INFO:teuthology.orchestra.run.smithi031.stdout:(6/32): python3-ceph-argparse-19.0.0-2515.g9680 776 kB/s | 45 kB 00:00 2024-05-08T08:22:07.981 INFO:teuthology.orchestra.run.smithi031.stdout:(7/32): python3-ceph-common-19.0.0-2515.g96805d 2.3 MB/s | 135 kB 00:00 2024-05-08T08:22:08.031 INFO:teuthology.orchestra.run.smithi031.stdout:(8/32): python3-cephfs-19.0.0-2515.g96805dcf.el 3.2 MB/s | 162 kB 00:00 2024-05-08T08:22:08.165 INFO:teuthology.orchestra.run.smithi031.stdout:(9/32): python3-rados-19.0.0-2515.g96805dcf.el9 2.4 MB/s | 321 kB 00:00 2024-05-08T08:22:08.249 INFO:teuthology.orchestra.run.smithi031.stdout:(10/32): librgw2-19.0.0-2515.g96805dcf.el9.x86_ 12 MB/s | 5.3 MB 00:00 2024-05-08T08:22:08.274 INFO:teuthology.orchestra.run.smithi031.stdout:(11/32): python3-rbd-19.0.0-2515.g96805dcf.el9. 2.7 MB/s | 298 kB 00:00 2024-05-08T08:22:08.299 INFO:teuthology.orchestra.run.smithi031.stdout:(12/32): python3-rgw-19.0.0-2515.g96805dcf.el9. 1.9 MB/s | 99 kB 00:00 2024-05-08T08:22:08.449 INFO:teuthology.orchestra.run.smithi031.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 611 kB/s | 107 kB 00:00 2024-05-08T08:22:08.475 INFO:teuthology.orchestra.run.smithi031.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 915 kB/s | 160 kB 00:00 2024-05-08T08:22:08.500 INFO:teuthology.orchestra.run.smithi031.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-05-08T08:22:08.584 INFO:teuthology.orchestra.run.smithi031.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-08T08:22:08.617 INFO:teuthology.orchestra.run.smithi031.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-05-08T08:22:08.642 INFO:teuthology.orchestra.run.smithi031.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 723 kB/s | 42 kB 00:00 2024-05-08T08:22:08.676 INFO:teuthology.orchestra.run.smithi031.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-08T08:22:08.701 INFO:teuthology.orchestra.run.smithi031.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 613 kB/s | 36 kB 00:00 2024-05-08T08:22:08.726 INFO:teuthology.orchestra.run.smithi031.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 947 kB/s | 47 kB 00:00 2024-05-08T08:22:09.064 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 10 M 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrading: 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout: librados2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.4 M 2024-05-08T08:22:09.067 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.4 M 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 21 M 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 25 k 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 746 k 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T08:22:09.068 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 500 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2 x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 5.3 M 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 45 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 135 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 162 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 321 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 298 k 2024-05-08T08:22:09.069 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 99 k 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout:Installing weak dependencies: 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:22:09.070 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:09.071 INFO:teuthology.orchestra.run.smithi149.stdout:Install 30 Packages 2024-05-08T08:22:09.071 INFO:teuthology.orchestra.run.smithi149.stdout:Upgrade 2 Packages 2024-05-08T08:22:09.071 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:09.071 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 60 M 2024-05-08T08:22:09.071 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:22:09.327 INFO:teuthology.orchestra.run.smithi031.stdout:(22/32): ceph-common-19.0.0-2515.g96805dcf.el9. 6.1 MB/s | 21 MB 00:03 2024-05-08T08:22:09.385 INFO:teuthology.orchestra.run.smithi031.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 452 kB/s | 309 kB 00:00 2024-05-08T08:22:09.569 INFO:teuthology.orchestra.run.smithi031.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 107 kB/s | 26 kB 00:00 2024-05-08T08:22:09.719 INFO:teuthology.orchestra.run.smithi031.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 146 kB/s | 49 kB 00:00 2024-05-08T08:22:09.886 INFO:teuthology.orchestra.run.smithi031.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 213 kB/s | 67 kB 00:00 2024-05-08T08:22:10.013 INFO:teuthology.orchestra.run.smithi149.stdout:(1/32): ceph-base-19.0.0-2515.g96805dcf.el9.x86 7.1 MB/s | 5.4 MB 00:00 2024-05-08T08:22:10.072 INFO:teuthology.orchestra.run.smithi149.stdout:(2/32): ceph-selinux-19.0.0-2515.g96805dcf.el9. 433 kB/s | 25 kB 00:00 2024-05-08T08:22:10.197 INFO:teuthology.orchestra.run.smithi149.stdout:(3/32): libcephfs2-19.0.0-2515.g96805dcf.el9.x8 5.8 MB/s | 746 kB 00:00 2024-05-08T08:22:10.281 INFO:teuthology.orchestra.run.smithi149.stdout:(4/32): libradosstriper1-19.0.0-2515.g96805dcf. 5.9 MB/s | 500 kB 00:00 2024-05-08T08:22:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 396 kB/s | 192 kB 00:00 2024-05-08T08:22:10.581 INFO:teuthology.orchestra.run.smithi149.stdout:(5/32): ceph-radosgw-19.0.0-2515.g96805dcf.el9. 7.7 MB/s | 10 MB 00:01 2024-05-08T08:22:10.656 INFO:teuthology.orchestra.run.smithi149.stdout:(6/32): python3-ceph-argparse-19.0.0-2515.g9680 602 kB/s | 45 kB 00:00 2024-05-08T08:22:10.732 INFO:teuthology.orchestra.run.smithi149.stdout:(7/32): python3-ceph-common-19.0.0-2515.g96805d 1.8 MB/s | 135 kB 00:00 2024-05-08T08:22:10.770 INFO:teuthology.orchestra.run.smithi031.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 799 kB/s | 839 kB 00:01 2024-05-08T08:22:10.799 INFO:teuthology.orchestra.run.smithi149.stdout:(8/32): librgw2-19.0.0-2515.g96805dcf.el9.x86_6 10 MB/s | 5.3 MB 00:00 2024-05-08T08:22:10.824 INFO:teuthology.orchestra.run.smithi149.stdout:(9/32): python3-cephfs-19.0.0-2515.g96805dcf.el 1.7 MB/s | 162 kB 00:00 2024-05-08T08:22:10.857 INFO:teuthology.orchestra.run.smithi149.stdout:(10/32): python3-rados-19.0.0-2515.g96805dcf.el 5.4 MB/s | 321 kB 00:00 2024-05-08T08:22:10.891 INFO:teuthology.orchestra.run.smithi149.stdout:(11/32): python3-rbd-19.0.0-2515.g96805dcf.el9. 4.4 MB/s | 298 kB 00:00 2024-05-08T08:22:10.916 INFO:teuthology.orchestra.run.smithi149.stdout:(12/32): python3-rgw-19.0.0-2515.g96805dcf.el9. 1.7 MB/s | 99 kB 00:00 2024-05-08T08:22:11.067 INFO:teuthology.orchestra.run.smithi149.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 610 kB/s | 107 kB 00:00 2024-05-08T08:22:11.134 INFO:teuthology.orchestra.run.smithi149.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 738 kB/s | 160 kB 00:00 2024-05-08T08:22:11.159 INFO:teuthology.orchestra.run.smithi149.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-05-08T08:22:11.195 INFO:teuthology.orchestra.run.smithi031.stdout:(29/32): librados2-19.0.0-2515.g96805dcf.el9.x8 8.0 MB/s | 3.4 MB 00:00 2024-05-08T08:22:11.267 INFO:teuthology.orchestra.run.smithi149.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-05-08T08:22:11.293 INFO:teuthology.orchestra.run.smithi149.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-05-08T08:22:11.404 INFO:teuthology.orchestra.run.smithi031.stdout:(30/32): librbd1-19.0.0-2515.g96805dcf.el9.x86_ 15 MB/s | 3.1 MB 00:00 2024-05-08T08:22:11.485 INFO:teuthology.orchestra.run.smithi149.stdout:(18/32): ceph-common-19.0.0-2515.g96805dcf.el9. 9.4 MB/s | 21 MB 00:02 2024-05-08T08:22:11.510 INFO:teuthology.orchestra.run.smithi149.stdout:(19/32): python3-prettytable-0.7.2-27.el9.noarc 174 kB/s | 42 kB 00:00 2024-05-08T08:22:11.535 INFO:teuthology.orchestra.run.smithi149.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 911 kB/s | 221 kB 00:00 2024-05-08T08:22:11.569 INFO:teuthology.orchestra.run.smithi149.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-05-08T08:22:11.619 INFO:teuthology.orchestra.run.smithi149.stdout:(22/32): qatlib-service-24.02.0-1.el9.x86_64.rp 268 kB/s | 36 kB 00:00 2024-05-08T08:22:11.694 INFO:teuthology.orchestra.run.smithi149.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 347 kB/s | 26 kB 00:00 2024-05-08T08:22:11.745 INFO:teuthology.orchestra.run.smithi149.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.4 MB/s | 309 kB 00:00 2024-05-08T08:22:11.795 INFO:teuthology.orchestra.run.smithi149.stdout:(25/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-05-08T08:22:11.853 INFO:teuthology.orchestra.run.smithi149.stdout:(26/32): liboath-2.6.7-2.el9.x86_64.rpm 306 kB/s | 49 kB 00:00 2024-05-08T08:22:11.937 INFO:teuthology.orchestra.run.smithi031.stdout:(31/32): libarrow-9.0.0-11.el9.x86_64.rpm 1.4 MB/s | 4.4 MB 00:03 2024-05-08T08:22:11.962 INFO:teuthology.orchestra.run.smithi149.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-08T08:22:12.129 INFO:teuthology.orchestra.run.smithi031.stdout:(32/32): thrift-0.15.0-2.el9.x86_64.rpm 921 kB/s | 1.6 MB 00:01 2024-05-08T08:22:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.9 MB/s | 839 kB 00:00 2024-05-08T08:22:12.232 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:22:12.232 INFO:teuthology.orchestra.run.smithi031.stdout:Total 8.9 MB/s | 60 MB 00:06 2024-05-08T08:22:12.242 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:22:12.271 INFO:teuthology.orchestra.run.smithi149.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-05-08T08:22:12.283 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:22:12.283 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:22:12.588 INFO:teuthology.orchestra.run.smithi149.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-05-08T08:22:12.930 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:22:12.931 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:22:12.947 INFO:teuthology.orchestra.run.smithi149.stdout:(31/32): librados2-19.0.0-2515.g96805dcf.el9.x8 4.7 MB/s | 3.4 MB 00:00 2024-05-08T08:22:12.997 INFO:teuthology.orchestra.run.smithi149.stdout:(32/32): librbd1-19.0.0-2515.g96805dcf.el9.x86_ 4.3 MB/s | 3.1 MB 00:00 2024-05-08T08:22:12.999 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:22:13.000 INFO:teuthology.orchestra.run.smithi149.stdout:Total 15 MB/s | 60 MB 00:03 2024-05-08T08:22:13.107 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:22:13.143 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:22:13.143 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:22:13.757 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:22:13.757 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:22:14.358 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:22:14.439 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T08:22:14.476 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T08:22:14.501 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T08:22:14.523 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:22:14.557 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:22:14.751 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:22:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:22:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:22:15.169 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T08:22:15.232 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T08:22:15.260 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T08:22:15.277 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:22:15.294 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T08:22:15.316 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:22:15.341 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T08:22:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:22:15.443 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 8/34 2024-05-08T08:22:15.487 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:22:15.527 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:22:15.550 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:22:15.578 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T08:22:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T08:22:15.623 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 11/34 2024-05-08T08:22:15.693 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:22:15.714 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:22:15.762 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:22:15.788 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T08:22:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T08:22:15.938 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T08:22:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T08:22:15.980 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T08:22:16.020 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T08:22:16.037 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:22:16.049 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T08:22:16.082 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T08:22:16.112 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:22:16.202 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 8/34 2024-05-08T08:22:16.230 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:22:16.293 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 9/34 2024-05-08T08:22:16.322 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T08:22:16.366 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 11/34 2024-05-08T08:22:16.423 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T08:22:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:22:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:22:16.461 INFO:teuthology.orchestra.run.smithi031.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:22:16.522 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 13/34 2024-05-08T08:22:16.553 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:22:16.556 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T08:22:16.592 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 23/34 2024-05-08T08:22:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T08:22:16.665 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T08:22:16.690 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T08:22:16.732 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T08:22:16.788 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T08:22:16.818 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T08:22:16.850 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T08:22:17.127 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T08:22:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:22:17.168 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T08:22:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:22:17.214 INFO:teuthology.orchestra.run.smithi149.stdout: Upgrading : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:22:17.289 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 27/34 2024-05-08T08:22:17.304 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 22/34 2024-05-08T08:22:17.316 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 28/34 2024-05-08T08:22:17.335 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 23/34 2024-05-08T08:22:17.371 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T08:22:17.844 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T08:22:17.883 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:22:17.954 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 26/34 2024-05-08T08:22:18.031 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 27/34 2024-05-08T08:22:18.061 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 28/34 2024-05-08T08:22:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:22:19.913 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:22:20.313 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:22:20.356 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:22:20.521 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:22:20.574 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:22:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:22:20.668 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T08:22:20.668 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T08:22:20.668 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:20.703 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:22:20.911 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 29/34 2024-05-08T08:22:20.960 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:22:21.052 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 30/34 2024-05-08T08:22:21.052 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T08:22:21.052 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T08:22:21.052 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:21.086 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /sys 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /proc 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /mnt 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /var/tmp 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /home 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /root 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout:skipping the directory /tmp 2024-05-08T08:22:30.682 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /sys 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /proc 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /mnt 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /var/tmp 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /home 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /root 2024-05-08T08:22:30.837 INFO:teuthology.orchestra.run.smithi031.stdout:skipping the directory /tmp 2024-05-08T08:22:30.838 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:31.648 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:22:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:22:31.674 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:22:31.674 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T08:22:31.674 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:22:31.674 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:22:31.674 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:31.676 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:22:31.758 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:22:31.759 INFO:teuthology.orchestra.run.smithi149.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:22:31.809 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:22:31.835 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 32/34 2024-05-08T08:22:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:22:31.835 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T08:22:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:22:31.836 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T08:22:31.836 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:31.837 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:22:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T08:22:31.937 INFO:teuthology.orchestra.run.smithi031.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:22:32.944 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:22:32.944 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 1/34 2024-05-08T08:22:32.944 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2/34 2024-05-08T08:22:32.944 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 3/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 4/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 5/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 6/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 8/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 9/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 10/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 11/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 13/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T08:22:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T08:22:32.946 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T08:22:32.946 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T08:22:32.946 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T08:22:32.946 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T08:22:32.946 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T08:22:32.946 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T08:22:32.947 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T08:22:32.947 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T08:22:32.947 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T08:22:32.947 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T08:22:32.949 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T08:22:32.949 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T08:22:32.949 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T08:22:32.949 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:22:32.949 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T08:22:32.949 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 33/34 2024-05-08T08:22:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:22:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 1/34 2024-05-08T08:22:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2/34 2024-05-08T08:22:33.093 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 3/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 4/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 5/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_6 6/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 7/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9. 8/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x8 9/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 10/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 11/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 12/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 13/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T08:22:33.094 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T08:22:33.095 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T08:22:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T08:22:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 31/34 2024-05-08T08:22:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T08:22:33.096 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 33/34 2024-05-08T08:22:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:22:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout:Upgraded: 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T08:22:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:33.771 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout:Upgraded: 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-radosgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.874 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-selinux-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: librgw2-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T08:22:33.875 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-argparse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ceph-common-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:33.876 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:22:34.124 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-test 2024-05-08T08:22:34.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-test 2024-05-08T08:22:34.705 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:28 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:22:34.811 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:32 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:22:34.830 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:22:34.830 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:34.830 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:22:34.830 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:34.830 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:22:34.830 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-08T08:22:34.830 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout:Install 4 Packages 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 49 M 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 207 M 2024-05-08T08:22:34.831 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:22:34.937 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:22:34.938 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:22:34.938 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:34.938 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:22:34.938 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 48 M 2024-05-08T08:22:34.938 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 163 k 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout:Install 4 Packages 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:34.939 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 49 M 2024-05-08T08:22:34.940 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 207 M 2024-05-08T08:22:34.940 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:22:35.074 INFO:teuthology.orchestra.run.smithi031.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-05-08T08:22:35.104 INFO:teuthology.orchestra.run.smithi149.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.3 MB/s | 305 kB 00:00 2024-05-08T08:22:35.171 INFO:teuthology.orchestra.run.smithi149.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 954 kB/s | 64 kB 00:00 2024-05-08T08:22:35.183 INFO:teuthology.orchestra.run.smithi031.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T08:22:35.488 INFO:teuthology.orchestra.run.smithi149.stdout:(3/4): libcephsqlite-19.0.0-2515.g96805dcf.el9. 263 kB/s | 163 kB 00:00 2024-05-08T08:22:35.633 INFO:teuthology.orchestra.run.smithi031.stdout:(3/4): libcephsqlite-19.0.0-2515.g96805dcf.el9. 251 kB/s | 163 kB 00:00 2024-05-08T08:22:39.272 INFO:teuthology.orchestra.run.smithi149.stdout:(4/4): ceph-test-19.0.0-2515.g96805dcf.el9.x86_ 11 MB/s | 48 MB 00:04 2024-05-08T08:22:39.274 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:22:39.274 INFO:teuthology.orchestra.run.smithi149.stdout:Total 11 MB/s | 49 MB 00:04 2024-05-08T08:22:39.301 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:22:39.322 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:22:39.322 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:22:39.683 INFO:teuthology.orchestra.run.smithi031.stdout:(4/4): ceph-test-19.0.0-2515.g96805dcf.el9.x86_ 10 MB/s | 48 MB 00:04 2024-05-08T08:22:39.685 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:22:39.685 INFO:teuthology.orchestra.run.smithi031.stdout:Total 10 MB/s | 49 MB 00:04 2024-05-08T08:22:39.691 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:22:39.692 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:22:39.713 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:22:39.738 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:22:39.738 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:22:40.122 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:22:40.123 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:22:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:22:40.311 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T08:22:40.362 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T08:22:40.390 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:22:40.661 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:22:40.741 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T08:22:40.791 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T08:22:40.818 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:22:44.964 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:22:45.046 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:22:45.785 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 3/4 2024-05-08T08:22:45.860 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:22:47.372 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:22:47.372 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 1/4 2024-05-08T08:22:47.372 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2/4 2024-05-08T08:22:47.372 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:47.665 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:22:47.965 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph 2024-05-08T08:22:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 4/4 2024-05-08T08:22:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 1/4 2024-05-08T08:22:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2/4 2024-05-08T08:22:48.271 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T08:22:48.547 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:42 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:22:48.674 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-08T08:22:48.678 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T08:22:48.679 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T08:22:48.680 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout:Install 36 Packages 2024-05-08T08:22:48.681 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:48.682 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 30 M 2024-05-08T08:22:48.682 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 107 M 2024-05-08T08:22:48.682 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:22:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T08:22:48.685 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:48.685 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:22:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-test-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: libcephsqlite-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:22:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T08:22:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T08:22:48.686 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:48.686 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:22:48.958 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph 2024-05-08T08:22:49.515 INFO:teuthology.orchestra.run.smithi149.stdout:(1/36): ceph-19.0.0-2515.g96805dcf.el9.x86_64.r 20 kB/s | 6.6 kB 00:00 2024-05-08T08:22:49.542 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:00:47 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:22:49.671 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:22:49.673 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:49.673 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: ceph x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 6.6 k 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 2.4 M 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 1.5 M 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 249 k 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 4.7 M 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 17 M 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T08:22:49.674 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T08:22:49.675 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:49.676 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:22:49.677 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:22:49.677 INFO:teuthology.orchestra.run.smithi031.stdout:Install 36 Packages 2024-05-08T08:22:49.677 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:22:49.677 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 30 M 2024-05-08T08:22:49.677 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 107 M 2024-05-08T08:22:49.677 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:22:50.140 INFO:teuthology.orchestra.run.smithi149.stdout:(2/36): ceph-mds-19.0.0-2515.g96805dcf.el9.x86_ 2.5 MB/s | 2.4 MB 00:00 2024-05-08T08:22:50.291 INFO:teuthology.orchestra.run.smithi149.stdout:(3/36): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86_ 1.4 MB/s | 1.5 MB 00:01 2024-05-08T08:22:50.391 INFO:teuthology.orchestra.run.smithi149.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2515.g9680 2.4 MB/s | 249 kB 00:00 2024-05-08T08:22:50.592 INFO:teuthology.orchestra.run.smithi149.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-08T08:22:50.800 INFO:teuthology.orchestra.run.smithi149.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 5.9 MB/s | 1.2 MB 00:00 2024-05-08T08:22:50.850 INFO:teuthology.orchestra.run.smithi149.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-08T08:22:50.851 INFO:teuthology.orchestra.run.smithi031.stdout:(1/36): ceph-19.0.0-2515.g96805dcf.el9.x86_64.r 18 kB/s | 6.6 kB 00:00 2024-05-08T08:22:50.908 INFO:teuthology.orchestra.run.smithi149.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.3 MB/s | 135 kB 00:00 2024-05-08T08:22:50.976 INFO:teuthology.orchestra.run.smithi149.stdout:(9/36): ceph-mon-19.0.0-2515.g96805dcf.el9.x86_ 3.2 MB/s | 4.7 MB 00:01 2024-05-08T08:22:51.001 INFO:teuthology.orchestra.run.smithi149.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 1.3 MB/s | 125 kB 00:00 2024-05-08T08:22:51.059 INFO:teuthology.orchestra.run.smithi149.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 3.2 MB/s | 188 kB 00:00 2024-05-08T08:22:51.118 INFO:teuthology.orchestra.run.smithi149.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-08T08:22:51.143 INFO:teuthology.orchestra.run.smithi149.stdout:(13/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-05-08T08:22:51.169 INFO:teuthology.orchestra.run.smithi149.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 690 kB/s | 35 kB 00:00 2024-05-08T08:22:51.194 INFO:teuthology.orchestra.run.smithi149.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-08T08:22:51.220 INFO:teuthology.orchestra.run.smithi149.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-05-08T08:22:51.351 INFO:teuthology.orchestra.run.smithi031.stdout:(2/36): ceph-mgr-19.0.0-2515.g96805dcf.el9.x86_ 1.7 MB/s | 1.5 MB 00:00 2024-05-08T08:22:51.395 INFO:teuthology.orchestra.run.smithi149.stdout:(17/36): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 13 MB/s | 17 MB 00:01 2024-05-08T08:22:51.453 INFO:teuthology.orchestra.run.smithi149.stdout:(18/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 747 kB/s | 43 kB 00:00 2024-05-08T08:22:51.476 INFO:teuthology.orchestra.run.smithi031.stdout:(3/36): ceph-mds-19.0.0-2515.g96805dcf.el9.x86_ 2.4 MB/s | 2.4 MB 00:00 2024-05-08T08:22:51.479 INFO:teuthology.orchestra.run.smithi149.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 583 kB/s | 151 kB 00:00 2024-05-08T08:22:51.512 INFO:teuthology.orchestra.run.smithi149.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-08T08:22:51.546 INFO:teuthology.orchestra.run.smithi149.stdout:(21/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 324 kB/s | 11 kB 00:00 2024-05-08T08:22:51.571 INFO:teuthology.orchestra.run.smithi149.stdout:(22/36): python3-cherrypy-18.6.1-2.el9.noarch.r 3.8 MB/s | 358 kB 00:00 2024-05-08T08:22:51.596 INFO:teuthology.orchestra.run.smithi149.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 350 kB/s | 18 kB 00:00 2024-05-08T08:22:51.622 INFO:teuthology.orchestra.run.smithi149.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-05-08T08:22:51.647 INFO:teuthology.orchestra.run.smithi149.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-05-08T08:22:51.697 INFO:teuthology.orchestra.run.smithi149.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-05-08T08:22:51.722 INFO:teuthology.orchestra.run.smithi149.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 917 kB/s | 46 kB 00:00 2024-05-08T08:22:51.748 INFO:teuthology.orchestra.run.smithi149.stdout:(28/36): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-05-08T08:22:51.773 INFO:teuthology.orchestra.run.smithi149.stdout:(29/36): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-05-08T08:22:51.798 INFO:teuthology.orchestra.run.smithi149.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 326 kB/s | 16 kB 00:00 2024-05-08T08:22:51.823 INFO:teuthology.orchestra.run.smithi149.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-05-08T08:22:51.849 INFO:teuthology.orchestra.run.smithi149.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 725 kB/s | 36 kB 00:00 2024-05-08T08:22:51.860 INFO:teuthology.orchestra.run.smithi031.stdout:(4/36): ceph-mon-19.0.0-2515.g96805dcf.el9.x86_ 4.7 MB/s | 4.7 MB 00:01 2024-05-08T08:22:51.874 INFO:teuthology.orchestra.run.smithi149.stdout:(33/36): python3-webob-1.8.7-6.el9.noarch.rpm 4.5 MB/s | 230 kB 00:00 2024-05-08T08:22:51.907 INFO:teuthology.orchestra.run.smithi149.stdout:(34/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 7.1 MB/s | 427 kB 00:00 2024-05-08T08:22:51.933 INFO:teuthology.orchestra.run.smithi149.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 342 kB/s | 20 kB 00:00 2024-05-08T08:22:52.052 INFO:teuthology.orchestra.run.smithi031.stdout:(5/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-05-08T08:22:52.168 INFO:teuthology.orchestra.run.smithi031.stdout:(6/36): python3-cryptography-36.0.1-4.el9.x86_6 11 MB/s | 1.2 MB 00:00 2024-05-08T08:22:52.194 INFO:teuthology.orchestra.run.smithi031.stdout:(7/36): ceph-mgr-modules-core-19.0.0-2515.g9680 347 kB/s | 249 kB 00:00 2024-05-08T08:22:52.199 INFO:teuthology.orchestra.run.smithi149.stdout:(36/36): lua-devel-5.4.4-4.el9.x86_64.rpm 22 kB/s | 22 kB 00:01 2024-05-08T08:22:52.203 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:22:52.203 INFO:teuthology.orchestra.run.smithi149.stdout:Total 8.5 MB/s | 30 MB 00:03 2024-05-08T08:22:52.227 INFO:teuthology.orchestra.run.smithi031.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-05-08T08:22:52.278 INFO:teuthology.orchestra.run.smithi031.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.5 MB/s | 125 kB 00:00 2024-05-08T08:22:52.345 INFO:teuthology.orchestra.run.smithi031.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 3.1 MB/s | 215 kB 00:00 2024-05-08T08:22:52.345 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:22:52.370 INFO:teuthology.orchestra.run.smithi031.stdout:(11/36): python3-pycparser-2.20-6.el9.noarch.rp 769 kB/s | 135 kB 00:00 2024-05-08T08:22:52.385 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:22:52.386 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:22:52.537 INFO:teuthology.orchestra.run.smithi031.stdout:(12/36): ceph-osd-19.0.0-2515.g96805dcf.el9.x86 14 MB/s | 17 MB 00:01 2024-05-08T08:22:52.823 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:22:52.824 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:22:53.212 INFO:teuthology.orchestra.run.smithi031.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 51 kB/s | 35 kB 00:00 2024-05-08T08:22:53.329 INFO:teuthology.orchestra.run.smithi031.stdout:(14/36): lua-5.4.4-4.el9.x86_64.rpm 191 kB/s | 188 kB 00:00 2024-05-08T08:22:53.346 INFO:teuthology.orchestra.run.smithi031.stdout:(15/36): python3-mako-1.1.4-6.el9.noarch.rpm 176 kB/s | 172 kB 00:00 2024-05-08T08:22:53.646 INFO:teuthology.orchestra.run.smithi031.stdout:(16/36): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-05-08T08:22:53.971 INFO:teuthology.orchestra.run.smithi031.stdout:(17/36): python3-pytz-2021.1-5.el9.noarch.rpm 67 kB/s | 51 kB 00:00 2024-05-08T08:22:54.071 INFO:teuthology.orchestra.run.smithi031.stdout:(18/36): python3-toml-0.10.2-6.el9.noarch.rpm 56 kB/s | 42 kB 00:00 2024-05-08T08:22:54.106 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:22:54.177 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T08:22:54.197 INFO:teuthology.orchestra.run.smithi031.stdout:(19/36): luarocks-3.9.2-1.el9.noarch.rpm 274 kB/s | 151 kB 00:00 2024-05-08T08:22:54.237 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T08:22:54.272 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T08:22:54.345 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T08:22:54.347 INFO:teuthology.orchestra.run.smithi031.stdout:(20/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 116 kB/s | 43 kB 00:00 2024-05-08T08:22:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T08:22:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T08:22:54.447 INFO:teuthology.orchestra.run.smithi031.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-05-08T08:22:54.464 INFO:teuthology.orchestra.run.smithi031.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 91 kB/s | 11 kB 00:00 2024-05-08T08:22:54.513 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T08:22:54.548 INFO:teuthology.orchestra.run.smithi031.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 176 kB/s | 18 kB 00:00 2024-05-08T08:22:54.581 INFO:teuthology.orchestra.run.smithi031.stdout:(24/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 337 kB/s | 172 kB 00:00 2024-05-08T08:22:54.583 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T08:22:54.606 INFO:teuthology.orchestra.run.smithi031.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 163 kB/s | 23 kB 00:00 2024-05-08T08:22:54.623 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T08:22:54.658 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T08:22:54.658 INFO:teuthology.orchestra.run.smithi031.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 178 kB/s | 19 kB 00:00 2024-05-08T08:22:54.687 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T08:22:54.699 INFO:teuthology.orchestra.run.smithi031.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 168 kB/s | 20 kB 00:00 2024-05-08T08:22:54.722 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T08:22:54.724 INFO:teuthology.orchestra.run.smithi031.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 395 kB/s | 46 kB 00:00 2024-05-08T08:22:54.755 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T08:22:54.757 INFO:teuthology.orchestra.run.smithi031.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 790 kB/s | 79 kB 00:00 2024-05-08T08:22:54.795 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T08:22:54.831 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T08:22:54.833 INFO:teuthology.orchestra.run.smithi031.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2024-05-08T08:22:54.856 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T08:22:54.866 INFO:teuthology.orchestra.run.smithi031.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 826 kB/s | 90 kB 00:00 2024-05-08T08:22:54.887 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T08:22:54.978 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T08:22:55.059 INFO:teuthology.orchestra.run.smithi031.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 159 kB/s | 36 kB 00:00 2024-05-08T08:22:55.084 INFO:teuthology.orchestra.run.smithi031.stdout:(33/36): python3-pecan-1.4.2-3.el9.noarch.rpm 708 kB/s | 272 kB 00:00 2024-05-08T08:22:55.101 INFO:teuthology.orchestra.run.smithi031.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 982 kB/s | 230 kB 00:00 2024-05-08T08:22:55.118 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T08:22:55.192 INFO:teuthology.orchestra.run.smithi031.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-05-08T08:22:55.204 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T08:22:55.274 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T08:22:55.309 INFO:teuthology.orchestra.run.smithi031.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.7 MB/s | 427 kB 00:00 2024-05-08T08:22:55.311 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:22:55.311 INFO:teuthology.orchestra.run.smithi031.stdout:Total 5.3 MB/s | 30 MB 00:05 2024-05-08T08:22:55.311 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T08:22:55.369 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T08:22:55.425 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T08:22:55.460 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:22:55.504 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:22:55.505 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:22:55.633 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T08:22:55.674 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T08:22:55.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T08:22:55.843 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T08:22:55.881 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T08:22:55.930 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:22:55.931 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:22:56.000 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T08:22:56.163 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 31/36 2024-05-08T08:22:56.199 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:22:56.225 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:22:56.225 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:22:56.226 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T08:22:56.226 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:22:56.226 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:22:56.226 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:57.217 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:22:57.279 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T08:22:57.316 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T08:22:57.348 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T08:22:57.423 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T08:22:57.463 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T08:22:57.496 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T08:22:57.592 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T08:22:57.667 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:22:57.669 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T08:22:57.695 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:22:57.695 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:22:57.695 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T08:22:57.695 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:22:57.695 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:22:57.695 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T08:22:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T08:22:57.780 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T08:22:57.823 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T08:22:57.856 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T08:22:57.897 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T08:22:57.941 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T08:22:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T08:22:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T08:22:58.097 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T08:22:58.142 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:22:58.166 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:22:58.167 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:22:58.167 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T08:22:58.167 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:22:58.167 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:22:58.167 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T08:22:58.323 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T08:22:58.382 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T08:22:58.401 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:22:58.420 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T08:22:58.425 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:22:58.425 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:22:58.425 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T08:22:58.425 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:22:58.425 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:22:58.425 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:22:58.478 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T08:22:58.498 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:22:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T08:22:58.764 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T08:22:58.815 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T08:22:58.896 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T08:22:58.994 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T08:22:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T08:22:59.169 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T08:22:59.335 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 31/36 2024-05-08T08:22:59.357 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:22:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 32/36 2024-05-08T08:22:59.385 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:22:59.385 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T08:22:59.385 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:22:59.385 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T08:22:59.385 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 3/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 4/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 5/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 6/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T08:23:00.291 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T08:23:00.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T08:23:00.293 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T08:23:00.293 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T08:23:00.294 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T08:23:00.295 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T08:23:00.295 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T08:23:00.900 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:23:00.929 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 33/36 2024-05-08T08:23:00.929 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:23:00.929 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T08:23:00.929 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:23:00.929 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T08:23:00.929 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:01.168 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T08:23:01.168 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:01.168 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:01.168 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T08:23:01.169 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T08:23:01.170 INFO:teuthology.orchestra.run.smithi149.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T08:23:01.171 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:01.172 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:01.400 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:23:01.426 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 34/36 2024-05-08T08:23:01.426 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:23:01.426 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T08:23:01.426 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:23:01.426 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T08:23:01.427 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:01.540 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-base 2024-05-08T08:23:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:23:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 35/36 2024-05-08T08:23:01.693 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:23:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T08:23:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:23:01.694 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T08:23:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:01.759 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:23:02.115 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:56 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:02.196 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:02.239 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:02.240 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:02.240 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:02.308 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install cephadm 2024-05-08T08:23:02.880 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:56 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:03.004 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:03.004 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 763 k 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 769 k 2024-05-08T08:23:03.005 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:03.467 INFO:teuthology.orchestra.run.smithi149.stdout:cephadm-19.0.0-2515.g96805dcf.el9.noarch.rpm 1.6 MB/s | 763 kB 00:00 2024-05-08T08:23:03.468 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:03.468 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.6 MB/s | 763 kB 00:00 2024-05-08T08:23:03.468 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:03.473 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:03.473 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:03.479 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:03.479 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 36/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 1/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 3/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 4/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 5/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9. 6/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T08:23:03.527 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T08:23:03.528 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T08:23:03.529 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T08:23:03.529 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T08:23:03.529 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T08:23:03.529 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T08:23:03.530 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T08:23:03.530 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T08:23:03.530 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T08:23:03.530 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T08:23:03.530 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T08:23:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:03.991 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:04.078 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mds-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:04.278 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mon-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-osd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T08:23:04.279 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T08:23:04.280 INFO:teuthology.orchestra.run.smithi031.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T08:23:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T08:23:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T08:23:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T08:23:04.281 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:04.281 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:04.456 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:04.613 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-base 2024-05-08T08:23:04.731 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:04.731 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:04.731 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:04.731 INFO:teuthology.orchestra.run.smithi149.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:04.731 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:04.731 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:04.939 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T08:23:05.201 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:03 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:05.284 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-base-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:05.328 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:05.329 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:05.398 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install cephadm 2024-05-08T08:23:05.517 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:00:59 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:05.641 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:05.642 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 145 k 2024-05-08T08:23:05.643 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 435 k 2024-05-08T08:23:05.643 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:05.983 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:03 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:05.997 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-immutable-object-cache-19.0.0-2515.g96805d 409 kB/s | 145 kB 00:00 2024-05-08T08:23:05.998 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:05.998 INFO:teuthology.orchestra.run.smithi149.stdout:Total 407 kB/s | 145 kB 00:00 2024-05-08T08:23:05.998 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:06.006 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:06.006 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:06.062 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:06.063 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:06.110 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:06.110 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:06.110 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 763 k 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:06.111 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 763 k 2024-05-08T08:23:06.112 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 769 k 2024-05-08T08:23:06.112 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:06.315 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:23:06.340 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:23:06.340 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:23:06.341 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T08:23:06.341 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:06.541 INFO:teuthology.orchestra.run.smithi031.stdout:cephadm-19.0.0-2515.g96805dcf.el9.noarch.rpm 1.7 MB/s | 763 kB 00:00 2024-05-08T08:23:06.541 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:06.542 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.7 MB/s | 763 kB 00:00 2024-05-08T08:23:06.542 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:06.548 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:06.548 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:06.554 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:06.554 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:06.629 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:07.091 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:07.289 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:23:07.289 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:07.289 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:07.289 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:07.289 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:07.289 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:07.522 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:07.534 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr 2024-05-08T08:23:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/1 2024-05-08T08:23:07.794 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:07.794 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:07.795 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:07.795 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:08.028 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T08:23:08.116 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:02 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:08.203 INFO:teuthology.orchestra.run.smithi149.stdout:Package ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:08.242 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:08.242 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:08.242 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:08.314 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T08:23:08.622 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:06 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:08.750 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:08.751 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:08.751 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-05-08T08:23:08.751 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:08.751 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:08.751 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 145 k 2024-05-08T08:23:08.751 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:08.751 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:08.752 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:08.752 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-05-08T08:23:08.752 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:08.752 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 145 k 2024-05-08T08:23:08.752 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 435 k 2024-05-08T08:23:08.752 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:08.888 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:02 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:08.990 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-immutable-object-cache-19.0.0-2515.g96805d 609 kB/s | 145 kB 00:00 2024-05-08T08:23:08.991 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:08.991 INFO:teuthology.orchestra.run.smithi031.stdout:Total 605 kB/s | 145 kB 00:00 2024-05-08T08:23:08.991 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:08.999 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:08.999 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:09.010 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:09.011 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T08:23:09.012 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout:Install 12 Packages 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:09.013 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 8.6 M 2024-05-08T08:23:09.014 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 103 M 2024-05-08T08:23:09.014 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:09.059 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:09.059 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:09.217 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:09.266 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:23:09.293 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:23:09.293 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:23:09.294 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T08:23:09.294 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:09.718 INFO:teuthology.orchestra.run.smithi149.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2515.g96 93 kB/s | 32 kB 00:00 2024-05-08T08:23:09.743 INFO:teuthology.orchestra.run.smithi149.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2515.g968 45 kB/s | 16 kB 00:00 2024-05-08T08:23:09.802 INFO:teuthology.orchestra.run.smithi149.stdout:(3/12): ceph-mgr-dashboard-19.0.0-2515.g96805dc 8.3 MB/s | 3.5 MB 00:00 2024-05-08T08:23:10.061 INFO:teuthology.orchestra.run.smithi149.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 849 kB/s | 269 kB 00:00 2024-05-08T08:23:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf. 1/1 2024-05-08T08:23:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:10.210 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-immutable-object-cache-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:10.210 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:10.210 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:10.236 INFO:teuthology.orchestra.run.smithi149.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.0 MB/s | 1.0 MB 00:00 2024-05-08T08:23:10.361 INFO:teuthology.orchestra.run.smithi149.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 156 kB/s | 19 kB 00:00 2024-05-08T08:23:10.386 INFO:teuthology.orchestra.run.smithi149.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 1.4 MB/s | 863 kB 00:00 2024-05-08T08:23:10.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr 2024-05-08T08:23:10.487 INFO:teuthology.orchestra.run.smithi149.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.3 MB/s | 548 kB 00:00 2024-05-08T08:23:10.620 INFO:teuthology.orchestra.run.smithi149.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 619 kB/s | 144 kB 00:00 2024-05-08T08:23:10.645 INFO:teuthology.orchestra.run.smithi149.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 195 kB/s | 31 kB 00:00 2024-05-08T08:23:10.704 INFO:teuthology.orchestra.run.smithi149.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 5.9 MB/s | 2.0 MB 00:00 2024-05-08T08:23:10.729 INFO:teuthology.orchestra.run.smithi149.stdout:(12/12): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-05-08T08:23:10.732 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:10.732 INFO:teuthology.orchestra.run.smithi149.stdout:Total 5.0 MB/s | 8.6 MB 00:01 2024-05-08T08:23:10.780 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:10.797 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:10.797 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:10.976 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:10.976 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:11.025 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:09 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:11.108 INFO:teuthology.orchestra.run.smithi031.stdout:Package ceph-mgr-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:11.153 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:11.154 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:11.154 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:11.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T08:23:11.589 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T08:23:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T08:23:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T08:23:11.784 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T08:23:11.816 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:09 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:11.886 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T08:23:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 3.5 M 2024-05-08T08:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:23:11.945 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 32 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 16 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-05-08T08:23:11.947 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:11.947 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 8.6 M 2024-05-08T08:23:11.947 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 103 M 2024-05-08T08:23:11.947 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:12.020 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T08:23:12.113 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T08:23:12.207 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T08:23:12.244 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:23:12.284 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 10/12 2024-05-08T08:23:12.519 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2515.g96 123 kB/s | 32 kB 00:00 2024-05-08T08:23:12.544 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2515.g968 58 kB/s | 16 kB 00:00 2024-05-08T08:23:12.853 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): ceph-mgr-dashboard-19.0.0-2515.g96805dc 6.0 MB/s | 3.5 MB 00:00 2024-05-08T08:23:13.128 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 461 kB/s | 269 kB 00:00 2024-05-08T08:23:13.245 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 4.6 MB/s | 548 kB 00:00 2024-05-08T08:23:13.279 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 583 kB/s | 19 kB 00:00 2024-05-08T08:23:13.329 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 11/12 2024-05-08T08:23:13.359 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:23:13.362 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): python3-grpcio-1.46.7-10.el9.x86_64.rpm 24 MB/s | 2.0 MB 00:00 2024-05-08T08:23:13.396 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): protobuf-3.14.0-13.el9.x86_64.rpm 1.2 MB/s | 1.0 MB 00:00 2024-05-08T08:23:13.421 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.4 MB/s | 144 kB 00:00 2024-05-08T08:23:13.455 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 5.5 MB/s | 188 kB 00:00 2024-05-08T08:23:13.472 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): python3-repoze-lru-0.7-16.el9.noarch.r 407 kB/s | 31 kB 00:00 2024-05-08T08:23:13.755 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 957 kB/s | 863 kB 00:00 2024-05-08T08:23:13.757 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:13.758 INFO:teuthology.orchestra.run.smithi031.stdout:Total 4.8 MB/s | 8.6 MB 00:01 2024-05-08T08:23:13.812 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:13.831 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:13.831 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:14.015 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:14.623 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:14.690 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T08:23:14.725 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T08:23:14.734 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:23:14.734 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 1/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 2/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 3/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T08:23:14.735 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T08:23:14.773 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T08:23:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T08:23:14.898 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T08:23:15.028 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T08:23:15.062 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T08:23:15.062 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:15.062 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:15.062 INFO:teuthology.orchestra.run.smithi149.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T08:23:15.062 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:15.062 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T08:23:15.063 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:15.064 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:15.138 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T08:23:15.209 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T08:23:15.244 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:23:15.276 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 10/12 2024-05-08T08:23:15.292 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T08:23:15.876 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:09 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:16.008 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:16.009 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-08T08:23:16.009 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:23:16.009 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-08T08:23:16.009 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:16.009 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T08:23:16.010 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:16.011 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:16.011 INFO:teuthology.orchestra.run.smithi149.stdout:======================================================================================= 2024-05-08T08:23:16.011 INFO:teuthology.orchestra.run.smithi149.stdout:Install 12 Packages 2024-05-08T08:23:16.011 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:16.011 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 38 M 2024-05-08T08:23:16.011 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 209 M 2024-05-08T08:23:16.011 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:16.379 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 11/12 2024-05-08T08:23:16.410 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:23:16.581 INFO:teuthology.orchestra.run.smithi149.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 902 kB/s | 195 kB 00:00 2024-05-08T08:23:16.698 INFO:teuthology.orchestra.run.smithi149.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.4 MB/s | 803 kB 00:00 2024-05-08T08:23:17.424 INFO:teuthology.orchestra.run.smithi149.stdout:(3/12): ceph-mgr-diskprediction-local-19.0.0-25 7.0 MB/s | 7.4 MB 00:01 2024-05-08T08:23:17.449 INFO:teuthology.orchestra.run.smithi149.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 38 kB/s | 33 kB 00:00 2024-05-08T08:23:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 12/12 2024-05-08T08:23:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el 1/12 2024-05-08T08:23:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noa 2/12 2024-05-08T08:23:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9 3/12 2024-05-08T08:23:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T08:23:17.655 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T08:23:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T08:23:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T08:23:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T08:23:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T08:23:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T08:23:17.656 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T08:23:17.966 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-grafana-dashboards-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-prometheus-alerts-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T08:23:17.967 INFO:teuthology.orchestra.run.smithi031.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T08:23:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T08:23:17.968 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:17.968 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:18.166 INFO:teuthology.orchestra.run.smithi149.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 25 kB/s | 18 kB 00:00 2024-05-08T08:23:18.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T08:23:18.316 INFO:teuthology.orchestra.run.smithi149.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 1.8 MB/s | 3.0 MB 00:01 2024-05-08T08:23:18.341 INFO:teuthology.orchestra.run.smithi149.stdout:(7/12): openblas-0.3.21-2.el9.x86_64.rpm 39 kB/s | 35 kB 00:00 2024-05-08T08:23:18.781 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:16 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:18.907 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:18.908 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-05-08T08:23:18.908 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:23:18.908 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 7.4 M 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T08:23:18.909 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:18.910 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:18.910 INFO:teuthology.orchestra.run.smithi031.stdout:======================================================================================= 2024-05-08T08:23:18.910 INFO:teuthology.orchestra.run.smithi031.stdout:Install 12 Packages 2024-05-08T08:23:18.910 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:18.911 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 38 M 2024-05-08T08:23:18.911 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 209 M 2024-05-08T08:23:18.911 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:19.557 INFO:teuthology.orchestra.run.smithi031.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2024-05-08T08:23:19.583 INFO:teuthology.orchestra.run.smithi149.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 193 kB/s | 244 kB 00:01 2024-05-08T08:23:19.615 INFO:teuthology.orchestra.run.smithi031.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 5.3 MB/s | 803 kB 00:00 2024-05-08T08:23:19.699 INFO:teuthology.orchestra.run.smithi031.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-05-08T08:23:19.749 INFO:teuthology.orchestra.run.smithi031.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-05-08T08:23:19.799 INFO:teuthology.orchestra.run.smithi031.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 705 kB/s | 35 kB 00:00 2024-05-08T08:23:20.025 INFO:teuthology.orchestra.run.smithi031.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 7.3 MB/s | 3.0 MB 00:00 2024-05-08T08:23:20.075 INFO:teuthology.orchestra.run.smithi031.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-05-08T08:23:20.083 INFO:teuthology.orchestra.run.smithi149.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 2.5 MB/s | 4.7 MB 00:01 2024-05-08T08:23:20.175 INFO:teuthology.orchestra.run.smithi149.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.8 MB/s | 5.1 MB 00:01 2024-05-08T08:23:20.250 INFO:teuthology.orchestra.run.smithi031.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 11 MB/s | 4.7 MB 00:00 2024-05-08T08:23:20.317 INFO:teuthology.orchestra.run.smithi031.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.7 MB/s | 250 kB 00:00 2024-05-08T08:23:20.417 INFO:teuthology.orchestra.run.smithi031.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-05-08T08:23:20.793 INFO:teuthology.orchestra.run.smithi031.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-2 5.6 MB/s | 7.4 MB 00:01 2024-05-08T08:23:20.834 INFO:teuthology.orchestra.run.smithi149.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 200 kB/s | 250 kB 00:01 2024-05-08T08:23:21.051 INFO:teuthology.orchestra.run.smithi031.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-05-08T08:23:21.052 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:21.052 INFO:teuthology.orchestra.run.smithi031.stdout:Total 18 MB/s | 38 MB 00:02 2024-05-08T08:23:21.416 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:21.443 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:21.444 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:21.890 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:21.890 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:21.925 INFO:teuthology.orchestra.run.smithi149.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 8.8 MB/s | 16 MB 00:01 2024-05-08T08:23:21.927 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:21.927 INFO:teuthology.orchestra.run.smithi149.stdout:Total 6.4 MB/s | 38 MB 00:05 2024-05-08T08:23:22.276 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:22.303 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:22.303 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:22.670 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:22.740 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T08:23:22.740 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:22.740 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:22.775 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:23:22.882 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:23:22.918 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T08:23:23.101 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T08:23:23.133 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T08:23:23.224 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T08:23:23.492 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:23.578 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T08:23:23.624 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:23:23.725 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:23:23.767 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T08:23:23.903 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T08:23:23.945 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T08:23:23.982 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T08:23:24.018 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T08:23:24.069 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T08:23:24.725 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T08:23:24.847 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T08:23:25.351 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:23:26.123 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:23:26.437 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T08:23:26.466 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:23:27.193 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T08:23:27.223 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:23:28.954 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 1/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:23:28.955 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T08:23:29.290 INFO:teuthology.orchestra.run.smithi031.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:29.291 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:29.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-rook 2024-05-08T08:23:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 12/12 2024-05-08T08:23:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805 1/12 2024-05-08T08:23:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T08:23:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T08:23:30.138 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T08:23:30.138 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T08:23:30.139 INFO:teuthology.orchestra.run.smithi149.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T08:23:30.140 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:30.140 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:30.309 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:28 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:30.437 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T08:23:30.439 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout:Install 13 Packages 2024-05-08T08:23:30.440 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:30.441 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 2.2 M 2024-05-08T08:23:30.441 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 26 M 2024-05-08T08:23:30.441 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:30.540 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-rook 2024-05-08T08:23:30.976 INFO:teuthology.orchestra.run.smithi031.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 83 kB/s | 26 kB 00:00 2024-05-08T08:23:31.018 INFO:teuthology.orchestra.run.smithi031.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 55 kB/s | 19 kB 00:00 2024-05-08T08:23:31.035 INFO:teuthology.orchestra.run.smithi031.stdout:(3/13): ceph-mgr-rook-19.0.0-2515.g96805dcf.el9 134 kB/s | 49 kB 00:00 2024-05-08T08:23:31.132 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:25 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:31.255 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 49 k 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T08:23:31.257 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout:Install 13 Packages 2024-05-08T08:23:31.258 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:31.259 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 2.2 M 2024-05-08T08:23:31.259 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 26 M 2024-05-08T08:23:31.259 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:31.385 INFO:teuthology.orchestra.run.smithi031.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 433 kB/s | 159 kB 00:00 2024-05-08T08:23:31.415 INFO:teuthology.orchestra.run.smithi149.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 381 kB/s | 26 kB 00:00 2024-05-08T08:23:31.427 INFO:teuthology.orchestra.run.smithi031.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 493 kB/s | 222 kB 00:00 2024-05-08T08:23:31.431 INFO:teuthology.orchestra.run.smithi149.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 228 kB/s | 19 kB 00:00 2024-05-08T08:23:31.486 INFO:teuthology.orchestra.run.smithi031.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 535 kB/s | 54 kB 00:00 2024-05-08T08:23:31.532 INFO:teuthology.orchestra.run.smithi149.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-05-08T08:23:31.544 INFO:teuthology.orchestra.run.smithi031.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 275 kB/s | 32 kB 00:00 2024-05-08T08:23:31.557 INFO:teuthology.orchestra.run.smithi149.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.1 MB/s | 222 kB 00:00 2024-05-08T08:23:31.594 INFO:teuthology.orchestra.run.smithi031.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 138 kB/s | 15 kB 00:00 2024-05-08T08:23:31.616 INFO:teuthology.orchestra.run.smithi149.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 916 kB/s | 54 kB 00:00 2024-05-08T08:23:31.619 INFO:teuthology.orchestra.run.smithi031.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 477 kB/s | 279 kB 00:00 2024-05-08T08:23:31.641 INFO:teuthology.orchestra.run.smithi149.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.5 MB/s | 279 kB 00:00 2024-05-08T08:23:31.704 INFO:teuthology.orchestra.run.smithi031.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.3 MB/s | 206 kB 00:00 2024-05-08T08:23:31.733 INFO:teuthology.orchestra.run.smithi149.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 276 kB/s | 32 kB 00:00 2024-05-08T08:23:31.758 INFO:teuthology.orchestra.run.smithi149.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 128 kB/s | 15 kB 00:00 2024-05-08T08:23:31.770 INFO:teuthology.orchestra.run.smithi031.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 1.3 MB/s | 90 kB 00:00 2024-05-08T08:23:31.795 INFO:teuthology.orchestra.run.smithi031.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 337 kB/s | 59 kB 00:00 2024-05-08T08:23:31.821 INFO:teuthology.orchestra.run.smithi031.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.5 MB/s | 1.0 MB 00:00 2024-05-08T08:23:31.821 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:31.821 INFO:teuthology.orchestra.run.smithi031.stdout:Total 1.6 MB/s | 2.2 MB 00:01 2024-05-08T08:23:31.825 INFO:teuthology.orchestra.run.smithi149.stdout:(9/13): python3-google-auth-2.29.0-1.el9.noarch 2.2 MB/s | 206 kB 00:00 2024-05-08T08:23:31.867 INFO:teuthology.orchestra.run.smithi149.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-08T08:23:31.884 INFO:teuthology.orchestra.run.smithi149.stdout:(11/13): ceph-mgr-rook-19.0.0-2515.g96805dcf.el 92 kB/s | 49 kB 00:00 2024-05-08T08:23:31.896 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:31.912 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:31.913 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:31.917 INFO:teuthology.orchestra.run.smithi149.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-05-08T08:23:31.951 INFO:teuthology.orchestra.run.smithi149.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 5.3 MB/s | 1.0 MB 00:00 2024-05-08T08:23:31.951 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:31.952 INFO:teuthology.orchestra.run.smithi149.stdout:Total 3.2 MB/s | 2.2 MB 00:00 2024-05-08T08:23:32.010 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:32.011 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:32.016 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:32.031 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:32.031 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:32.117 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:32.118 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:32.546 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:32.619 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:32.719 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T08:23:32.774 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T08:23:32.797 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T08:23:32.830 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T08:23:32.853 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T08:23:32.865 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T08:23:32.908 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T08:23:32.916 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T08:23:32.945 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T08:23:32.986 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T08:23:33.004 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T08:23:33.080 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T08:23:33.115 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T08:23:33.168 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T08:23:33.181 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T08:23:33.230 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T08:23:33.925 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T08:23:33.946 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T08:23:33.973 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T08:23:33.993 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T08:23:34.006 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T08:23:34.019 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T08:23:34.056 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T08:23:34.058 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T08:23:34.084 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:23:34.088 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 1/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T08:23:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T08:23:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T08:23:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T08:23:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T08:23:34.835 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 13/13 2024-05-08T08:23:34.835 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 1/13 2024-05-08T08:23:34.835 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T08:23:34.835 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T08:23:34.835 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T08:23:34.835 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T08:23:34.836 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T08:23:34.836 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T08:23:34.836 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T08:23:34.836 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T08:23:34.836 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T08:23:34.836 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T08:23:34.836 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T08:23:35.139 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T08:23:35.139 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:35.139 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:35.139 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:35.140 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-rook-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T08:23:35.176 INFO:teuthology.orchestra.run.smithi149.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T08:23:35.177 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T08:23:35.177 INFO:teuthology.orchestra.run.smithi149.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T08:23:35.177 INFO:teuthology.orchestra.run.smithi149.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T08:23:35.177 INFO:teuthology.orchestra.run.smithi149.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T08:23:35.177 INFO:teuthology.orchestra.run.smithi149.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T08:23:35.177 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:35.177 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:35.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T08:23:35.449 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T08:23:35.980 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:33 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:36.028 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:30 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:36.109 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T08:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout:Install 6 Packages 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:36.111 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 7.0 M 2024-05-08T08:23:36.112 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 33 M 2024-05-08T08:23:36.112 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:36.151 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 142 k 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T08:23:36.152 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout:Install 6 Packages 2024-05-08T08:23:36.153 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:36.154 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 7.0 M 2024-05-08T08:23:36.154 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 33 M 2024-05-08T08:23:36.154 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:36.628 INFO:teuthology.orchestra.run.smithi149.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-05-08T08:23:36.715 INFO:teuthology.orchestra.run.smithi031.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-05-08T08:23:36.907 INFO:teuthology.orchestra.run.smithi031.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2024-05-08T08:23:37.011 INFO:teuthology.orchestra.run.smithi149.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2515.g96805dcf.e 243 kB/s | 142 kB 00:00 2024-05-08T08:23:37.036 INFO:teuthology.orchestra.run.smithi149.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-05-08T08:23:37.078 INFO:teuthology.orchestra.run.smithi149.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 863 kB/s | 58 kB 00:00 2024-05-08T08:23:37.107 INFO:teuthology.orchestra.run.smithi031.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2515.g96805dcf.e 251 kB/s | 142 kB 00:00 2024-05-08T08:23:37.112 INFO:teuthology.orchestra.run.smithi149.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 677 kB/s | 51 kB 00:00 2024-05-08T08:23:37.216 INFO:teuthology.orchestra.run.smithi031.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 187 kB/s | 58 kB 00:00 2024-05-08T08:23:37.266 INFO:teuthology.orchestra.run.smithi031.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 994 kB/s | 548 kB 00:00 2024-05-08T08:23:37.441 INFO:teuthology.orchestra.run.smithi031.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 153 kB/s | 51 kB 00:00 2024-05-08T08:23:37.445 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:37.445 INFO:teuthology.orchestra.run.smithi031.stdout:Total 5.2 MB/s | 7.0 MB 00:01 2024-05-08T08:23:37.519 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:37.532 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:37.532 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:37.636 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:37.638 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:37.870 INFO:teuthology.orchestra.run.smithi149.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.1 MB/s | 6.0 MB 00:01 2024-05-08T08:23:37.872 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:37.872 INFO:teuthology.orchestra.run.smithi149.stdout:Total 4.1 MB/s | 7.0 MB 00:01 2024-05-08T08:23:37.944 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:37.953 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:37.955 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:37.955 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:38.052 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:38.057 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:38.123 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T08:23:38.186 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T08:23:38.374 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:38.544 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T08:23:38.597 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T08:23:38.737 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T08:23:38.833 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T08:23:38.903 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T08:23:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:23:39.129 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T08:23:39.216 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T08:23:39.284 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T08:23:39.309 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:23:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:23:39.632 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/6 2024-05-08T08:23:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T08:23:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T08:23:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T08:23:39.633 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T08:23:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T08:23:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:39.896 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:39.896 INFO:teuthology.orchestra.run.smithi031.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T08:23:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T08:23:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T08:23:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T08:23:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T08:23:39.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:39.897 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:40.107 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 6/6 2024-05-08T08:23:40.107 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 1/6 2024-05-08T08:23:40.107 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T08:23:40.107 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T08:23:40.107 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T08:23:40.108 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T08:23:40.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-fuse 2024-05-08T08:23:40.367 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T08:23:40.367 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-mgr-cephadm-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:40.368 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:40.611 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-fuse 2024-05-08T08:23:40.734 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:38 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:40.862 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T08:23:40.863 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-05-08T08:23:40.864 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 970 k 2024-05-08T08:23:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 2.8 M 2024-05-08T08:23:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:41.191 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:35 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:41.315 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:41.315 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 889 k 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout:Install 2 Packages 2024-05-08T08:23:41.316 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:41.317 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 970 k 2024-05-08T08:23:41.317 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 2.8 M 2024-05-08T08:23:41.317 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:41.530 INFO:teuthology.orchestra.run.smithi149.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 529 kB/s | 80 kB 00:00 2024-05-08T08:23:41.794 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 102 kB/s | 80 kB 00:00 2024-05-08T08:23:41.981 INFO:teuthology.orchestra.run.smithi149.stdout:(2/2): ceph-fuse-19.0.0-2515.g96805dcf.el9.x86_ 1.4 MB/s | 889 kB 00:00 2024-05-08T08:23:41.981 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:41.982 INFO:teuthology.orchestra.run.smithi149.stdout:Total 1.4 MB/s | 970 kB 00:00 2024-05-08T08:23:41.985 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-fuse-19.0.0-2515.g96805dcf.el9.x86_ 907 kB/s | 889 kB 00:00 2024-05-08T08:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:41.986 INFO:teuthology.orchestra.run.smithi031.stdout:Total 863 kB/s | 970 kB 00:01 2024-05-08T08:23:41.994 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:42.000 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:42.017 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:42.017 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:42.025 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:42.025 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:42.096 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:42.096 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:42.109 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:42.287 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:42.304 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:42.449 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T08:23:42.450 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T08:23:42.513 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:23:42.522 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:23:43.182 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:23:43.183 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/2 2024-05-08T08:23:43.184 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2/2 2024-05-08T08:23:43.184 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/2 2024-05-08T08:23:43.445 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T08:23:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:43.446 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T08:23:43.446 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:43.446 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:43.475 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T08:23:43.475 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:43.476 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:43.476 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T08:23:43.476 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:43.476 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:43.726 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install ceph-volume 2024-05-08T08:23:43.769 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install ceph-volume 2024-05-08T08:23:44.321 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:42 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:44.348 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:38 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:44.448 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:44.448 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repository Size 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:Installing dependencies: 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:Install 2 Packages 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 341 k 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 1.5 M 2024-05-08T08:23:44.449 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:44.477 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:44.477 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:44.477 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repository Size 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume noarch 2:19.0.0-2515.g96805dcf.el9 ceph-noarch 264 k 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout:Installing dependencies: 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout:Install 2 Packages 2024-05-08T08:23:44.478 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:44.479 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 341 k 2024-05-08T08:23:44.479 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 1.5 M 2024-05-08T08:23:44.479 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:44.944 INFO:teuthology.orchestra.run.smithi149.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 180 kB/s | 77 kB 00:00 2024-05-08T08:23:45.025 INFO:teuthology.orchestra.run.smithi031.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 181 kB/s | 77 kB 00:00 2024-05-08T08:23:45.136 INFO:teuthology.orchestra.run.smithi149.stdout:(2/2): ceph-volume-19.0.0-2515.g96805dcf.el9.no 425 kB/s | 264 kB 00:00 2024-05-08T08:23:45.136 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:45.136 INFO:teuthology.orchestra.run.smithi149.stdout:Total 518 kB/s | 341 kB 00:00 2024-05-08T08:23:45.148 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:45.157 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:45.158 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:45.158 INFO:teuthology.orchestra.run.smithi031.stdout:(2/2): ceph-volume-19.0.0-2515.g96805dcf.el9.no 470 kB/s | 264 kB 00:00 2024-05-08T08:23:45.158 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:45.159 INFO:teuthology.orchestra.run.smithi031.stdout:Total 480 kB/s | 341 kB 00:00 2024-05-08T08:23:45.171 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:45.182 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:45.182 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:45.191 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:45.192 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:45.219 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:45.220 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:45.336 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:45.370 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:45.526 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T08:23:45.557 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T08:23:45.563 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:23:45.578 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:23:45.578 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:23:45.578 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T08:23:45.579 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:45.585 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:23:45.603 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2/2 2024-05-08T08:23:45.603 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:23:45.603 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T08:23:45.603 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:46.292 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 1/2 2024-05-08T08:23:46.319 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 1/2 2024-05-08T08:23:46.593 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T08:23:46.593 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:46.594 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:46.594 INFO:teuthology.orchestra.run.smithi149.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:46.594 INFO:teuthology.orchestra.run.smithi149.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T08:23:46.594 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:46.594 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:46.645 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T08:23:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:46.646 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-volume-2:19.0.0-2515.g96805dcf.el9.noarch 2024-05-08T08:23:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T08:23:46.646 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:46.646 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:46.892 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados-devel 2024-05-08T08:23:46.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados-devel 2024-05-08T08:23:47.471 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:41 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:47.549 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:47.597 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:47.597 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:47.597 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-05-08T08:23:47.598 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:47.599 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 127 k 2024-05-08T08:23:47.599 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 456 k 2024-05-08T08:23:47.599 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:47.678 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:47.679 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:47.679 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-05-08T08:23:47.679 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:47.679 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 127 k 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 127 k 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 456 k 2024-05-08T08:23:47.680 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:47.890 INFO:teuthology.orchestra.run.smithi031.stdout:librados-devel-19.0.0-2515.g96805dcf.el9.x86_64 606 kB/s | 127 kB 00:00 2024-05-08T08:23:47.890 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:47.891 INFO:teuthology.orchestra.run.smithi031.stdout:Total 600 kB/s | 127 kB 00:00 2024-05-08T08:23:47.891 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:47.897 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:47.897 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:47.950 INFO:teuthology.orchestra.run.smithi149.stdout:librados-devel-19.0.0-2515.g96805dcf.el9.x86_64 361 kB/s | 127 kB 00:00 2024-05-08T08:23:47.951 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:47.951 INFO:teuthology.orchestra.run.smithi149.stdout:Total 360 kB/s | 127 kB 00:00 2024-05-08T08:23:47.951 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:47.957 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:47.957 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:47.966 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:47.969 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:48.020 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:48.020 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:48.144 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:48.241 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:48.298 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:48.674 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:48.755 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:48.952 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:48.952 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:48.953 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:48.953 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:49.024 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:49.024 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:49.024 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:49.025 INFO:teuthology.orchestra.run.smithi149.stdout: librados-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:49.025 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:49.025 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:49.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs2 2024-05-08T08:23:49.309 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs2 2024-05-08T08:23:49.835 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:49.895 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:43 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:49.918 INFO:teuthology.orchestra.run.smithi031.stdout:Package libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:49.962 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:49.963 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:49.963 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:49.976 INFO:teuthology.orchestra.run.smithi149.stdout:Package libcephfs2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:50.018 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:50.019 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:50.019 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:50.058 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install libcephfs-devel 2024-05-08T08:23:50.096 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install libcephfs-devel 2024-05-08T08:23:50.658 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:50.678 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:44 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:50.784 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 32 k 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 142 k 2024-05-08T08:23:50.785 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:50.801 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:50.801 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:50.801 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-05-08T08:23:50.801 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:50.801 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:50.801 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 32 k 2024-05-08T08:23:50.801 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:50.802 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:50.802 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:50.802 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-05-08T08:23:50.802 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:50.802 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 32 k 2024-05-08T08:23:50.803 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 142 k 2024-05-08T08:23:50.803 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:50.953 INFO:teuthology.orchestra.run.smithi031.stdout:libcephfs-devel-19.0.0-2515.g96805dcf.el9.x86_6 189 kB/s | 32 kB 00:00 2024-05-08T08:23:50.954 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:50.954 INFO:teuthology.orchestra.run.smithi031.stdout:Total 187 kB/s | 32 kB 00:00 2024-05-08T08:23:50.954 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:50.958 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:50.958 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:50.980 INFO:teuthology.orchestra.run.smithi149.stdout:libcephfs-devel-19.0.0-2515.g96805dcf.el9.x86_6 179 kB/s | 32 kB 00:00 2024-05-08T08:23:50.980 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:50.981 INFO:teuthology.orchestra.run.smithi149.stdout:Total 177 kB/s | 32 kB 00:00 2024-05-08T08:23:50.981 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:50.984 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:50.984 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:50.986 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:50.986 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:51.007 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:51.007 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:51.070 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:51.091 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:51.187 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:51.210 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:51.561 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:51.618 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:51.785 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:51.785 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:51.785 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:51.858 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:51.858 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:51.858 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:51.858 INFO:teuthology.orchestra.run.smithi149.stdout: libcephfs-devel-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:51.858 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:51.858 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:51.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librados2 2024-05-08T08:23:52.108 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librados2 2024-05-08T08:23:52.546 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:52.628 INFO:teuthology.orchestra.run.smithi031.stdout:Package librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:52.672 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:52.673 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:52.673 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:52.686 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:46 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:52.743 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install librbd1 2024-05-08T08:23:52.768 INFO:teuthology.orchestra.run.smithi149.stdout:Package librados2-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:52.810 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:52.811 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:52.811 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:52.875 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install librbd1 2024-05-08T08:23:53.341 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:51 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:53.425 INFO:teuthology.orchestra.run.smithi031.stdout:Package librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:53.460 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:53.469 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:53.470 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:53.470 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:53.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rados 2024-05-08T08:23:53.542 INFO:teuthology.orchestra.run.smithi149.stdout:Package librbd1-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:53.584 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:53.585 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:53.585 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:53.648 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rados 2024-05-08T08:23:54.127 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:54.212 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:54.221 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:54.255 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:54.256 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:54.256 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:54.302 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rados-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:54.326 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rgw 2024-05-08T08:23:54.343 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:54.344 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:54.344 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:54.416 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rgw 2024-05-08T08:23:54.923 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:54.998 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:55.008 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:55.053 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:55.054 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:55.054 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:55.080 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rgw-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:55.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-cephfs 2024-05-08T08:23:55.122 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:55.123 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:55.123 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:55.192 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-cephfs 2024-05-08T08:23:55.710 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:53 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:55.766 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:55.793 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:55.837 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:55.838 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:55.838 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:55.847 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-cephfs-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:55.889 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:55.890 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:55.890 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:55.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install python3-rbd 2024-05-08T08:23:55.954 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install python3-rbd 2024-05-08T08:23:56.501 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:54 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:56.531 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:56.586 INFO:teuthology.orchestra.run.smithi031.stdout:Package python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:56.611 INFO:teuthology.orchestra.run.smithi149.stdout:Package python3-rbd-2:19.0.0-2515.g96805dcf.el9.x86_64 is already installed. 2024-05-08T08:23:56.630 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:56.631 INFO:teuthology.orchestra.run.smithi031.stdout:Nothing to do. 2024-05-08T08:23:56.632 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:56.653 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:56.654 INFO:teuthology.orchestra.run.smithi149.stdout:Nothing to do. 2024-05-08T08:23:56.654 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:56.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-fuse 2024-05-08T08:23:56.715 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-fuse 2024-05-08T08:23:57.284 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:55 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:57.291 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:51 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:57.412 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:57.413 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:57.413 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-05-08T08:23:57.413 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:57.413 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:57.413 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-08T08:23:57.413 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:57.413 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:57.414 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:57.414 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-05-08T08:23:57.414 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:57.414 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 85 k 2024-05-08T08:23:57.414 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 231 k 2024-05-08T08:23:57.414 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 85 k 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 85 k 2024-05-08T08:23:57.415 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 231 k 2024-05-08T08:23:57.416 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:23:57.798 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-fuse-19.0.0-2515.g96805dcf.el9.x86_64.rpm 222 kB/s | 85 kB 00:00 2024-05-08T08:23:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:Total 221 kB/s | 85 kB 00:00 2024-05-08T08:23:57.799 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:23:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:23:57.810 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:23:57.833 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-fuse-19.0.0-2515.g96805dcf.el9.x86_64.rpm 204 kB/s | 85 kB 00:00 2024-05-08T08:23:57.834 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:23:57.834 INFO:teuthology.orchestra.run.smithi149.stdout:Total 203 kB/s | 85 kB 00:00 2024-05-08T08:23:57.834 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:23:57.842 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:23:57.842 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:23:57.874 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:23:57.874 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:23:57.906 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:23:57.906 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:23:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:23:58.053 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:23:58.148 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:58.155 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:58.502 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:58.526 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:58.765 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:23:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:58.765 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:58.765 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:23:58.789 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:23:58.789 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:58.789 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:23:58.789 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-fuse-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:23:58.789 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:58.789 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:23:58.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-mirror 2024-05-08T08:23:59.028 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-mirror 2024-05-08T08:23:59.586 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:01:57 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:23:59.616 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:53 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:23:59.714 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:23:59.715 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:59.715 INFO:teuthology.orchestra.run.smithi031.stdout: Package Arch Version Repo Size 2024-05-08T08:23:59.715 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:59.715 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:23:59.715 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:23:59.715 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:59.716 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:23:59.716 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:23:59.716 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-05-08T08:23:59.716 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:23:59.716 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 3.1 M 2024-05-08T08:23:59.716 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 13 M 2024-05-08T08:23:59.716 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:23:59.742 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout: Package Arch Version Repo Size 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 3.1 M 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-05-08T08:23:59.743 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:23:59.744 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 3.1 M 2024-05-08T08:23:59.744 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 13 M 2024-05-08T08:23:59.744 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:24:03.067 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-mirror-19.0.0-2515.g96805dcf.el9.x86_64.rpm 944 kB/s | 3.1 MB 00:03 2024-05-08T08:24:03.068 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:24:03.068 INFO:teuthology.orchestra.run.smithi031.stdout:Total 943 kB/s | 3.1 MB 00:03 2024-05-08T08:24:03.069 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:24:03.080 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:24:03.080 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:24:03.095 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-mirror-19.0.0-2515.g96805dcf.el9.x86_64.rpm 944 kB/s | 3.1 MB 00:03 2024-05-08T08:24:03.096 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:24:03.097 INFO:teuthology.orchestra.run.smithi149.stdout:Total 943 kB/s | 3.1 MB 00:03 2024-05-08T08:24:03.097 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:24:03.108 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:24:03.108 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:24:03.170 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:24:03.170 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:24:03.192 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:24:03.192 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:24:03.637 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:24:03.696 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:03.721 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:03.721 INFO:teuthology.orchestra.run.smithi149.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:24:03.721 INFO:teuthology.orchestra.run.smithi149.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T08:24:03.721 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:24:03.721 INFO:teuthology.orchestra.run.smithi149.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:24:03.722 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:03.741 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:24:03.810 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:03.836 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:03.837 INFO:teuthology.orchestra.run.smithi031.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T08:24:03.837 INFO:teuthology.orchestra.run.smithi031.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T08:24:03.837 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:24:03.837 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T08:24:03.837 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:04.830 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:04.830 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:04.830 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:24:04.830 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:24:04.830 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:04.830 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:24:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:04.933 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:04.933 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:24:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-mirror-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:24:04.934 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:04.934 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:24:05.083 DEBUG:teuthology.orchestra.run.smithi149:> sudo yum -y install rbd-nbd 2024-05-08T08:24:05.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo yum -y install rbd-nbd 2024-05-08T08:24:05.668 INFO:teuthology.orchestra.run.smithi149.stdout:Last metadata expiration check: 0:01:59 ago on Wed 08 May 2024 08:22:06 AM UTC. 2024-05-08T08:24:05.768 INFO:teuthology.orchestra.run.smithi031.stdout:Last metadata expiration check: 0:02:03 ago on Wed 08 May 2024 08:22:02 AM UTC. 2024-05-08T08:24:05.793 INFO:teuthology.orchestra.run.smithi149.stdout:Dependencies resolved. 2024-05-08T08:24:05.794 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout: Package Architecture Version Repository Size 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:Installing: 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction Summary 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:================================================================================ 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:Install 1 Package 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:Total download size: 171 k 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:Installed size: 494 k 2024-05-08T08:24:05.795 INFO:teuthology.orchestra.run.smithi149.stdout:Downloading Packages: 2024-05-08T08:24:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:Dependencies resolved. 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout: Package Architecture Version Repository Size 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:Installing: 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd x86_64 2:19.0.0-2515.g96805dcf.el9 ceph 171 k 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction Summary 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:================================================================================ 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:Install 1 Package 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:Total download size: 171 k 2024-05-08T08:24:05.897 INFO:teuthology.orchestra.run.smithi031.stdout:Installed size: 494 k 2024-05-08T08:24:05.898 INFO:teuthology.orchestra.run.smithi031.stdout:Downloading Packages: 2024-05-08T08:24:06.375 INFO:teuthology.orchestra.run.smithi149.stdout:rbd-nbd-19.0.0-2515.g96805dcf.el9.x86_64.rpm 296 kB/s | 171 kB 00:00 2024-05-08T08:24:06.375 INFO:teuthology.orchestra.run.smithi149.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:24:06.376 INFO:teuthology.orchestra.run.smithi149.stdout:Total 295 kB/s | 171 kB 00:00 2024-05-08T08:24:06.376 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction check 2024-05-08T08:24:06.383 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction check succeeded. 2024-05-08T08:24:06.384 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction test 2024-05-08T08:24:06.447 INFO:teuthology.orchestra.run.smithi149.stdout:Transaction test succeeded. 2024-05-08T08:24:06.448 INFO:teuthology.orchestra.run.smithi149.stdout:Running transaction 2024-05-08T08:24:06.469 INFO:teuthology.orchestra.run.smithi031.stdout:rbd-nbd-19.0.0-2515.g96805dcf.el9.x86_64.rpm 300 kB/s | 171 kB 00:00 2024-05-08T08:24:06.470 INFO:teuthology.orchestra.run.smithi031.stdout:-------------------------------------------------------------------------------- 2024-05-08T08:24:06.470 INFO:teuthology.orchestra.run.smithi031.stdout:Total 299 kB/s | 171 kB 00:00 2024-05-08T08:24:06.470 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction check 2024-05-08T08:24:06.479 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction check succeeded. 2024-05-08T08:24:06.479 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction test 2024-05-08T08:24:06.549 INFO:teuthology.orchestra.run.smithi031.stdout:Transaction test succeeded. 2024-05-08T08:24:06.550 INFO:teuthology.orchestra.run.smithi031.stdout:Running transaction 2024-05-08T08:24:06.610 INFO:teuthology.orchestra.run.smithi149.stdout: Preparing : 1/1 2024-05-08T08:24:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: Preparing : 1/1 2024-05-08T08:24:06.811 INFO:teuthology.orchestra.run.smithi031.stdout: Installing : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:07.138 INFO:teuthology.orchestra.run.smithi149.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:07.382 INFO:teuthology.orchestra.run.smithi149.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:07.382 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:07.382 INFO:teuthology.orchestra.run.smithi149.stdout:Installed: 2024-05-08T08:24:07.382 INFO:teuthology.orchestra.run.smithi149.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:24:07.382 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:07.382 INFO:teuthology.orchestra.run.smithi149.stdout:Complete! 2024-05-08T08:24:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: Verifying : rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 1/1 2024-05-08T08:24:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:07.485 INFO:teuthology.orchestra.run.smithi031.stdout:Installed: 2024-05-08T08:24:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: rbd-nbd-2:19.0.0-2515.g96805dcf.el9.x86_64 2024-05-08T08:24:07.485 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:07.485 INFO:teuthology.orchestra.run.smithi031.stdout:Complete! 2024-05-08T08:24:07.606 DEBUG:teuthology.parallel:result is None 2024-05-08T08:24:07.741 DEBUG:teuthology.parallel:result is None 2024-05-08T08:24:07.742 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:24:07.963 DEBUG:teuthology.orchestra.run.smithi031:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T08:24:07.987 INFO:teuthology.orchestra.run.smithi031.stdout:19.0.0-2515.g96805dcf.el9 2024-05-08T08:24:07.987 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-08T08:24:07.987 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-08T08:24:07.989 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=96805dcf909541c4ca9a1e90258de8ae5b6fa1bc 2024-05-08T08:24:08.240 DEBUG:teuthology.orchestra.run.smithi149:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T08:24:08.266 INFO:teuthology.orchestra.run.smithi149.stdout:19.0.0-2515.g96805dcf.el9 2024-05-08T08:24:08.266 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2515.g96805dcf.el9 2024-05-08T08:24:08.266 INFO:teuthology.task.install:The correct ceph version 19.0.0-2515.g96805dcf is installed. 2024-05-08T08:24:08.268 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T08:24:08.268 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:08.268 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T08:24:08.297 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:08.298 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T08:24:08.342 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T08:24:08.342 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:08.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T08:24:08.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T08:24:08.439 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:08.439 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T08:24:08.468 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T08:24:08.536 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T08:24:08.536 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:08.536 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T08:24:08.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T08:24:08.631 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:08.631 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T08:24:08.658 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T08:24:08.723 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T08:24:08.723 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:08.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T08:24:08.750 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T08:24:08.817 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:08.818 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T08:24:08.844 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T08:24:08.910 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T08:24:08.989 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T08:24:08.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-05-08T08:24:08.991 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 777 /var/log/ceph 2024-05-08T08:24:09.018 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T08:24:09.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T08:24:09.060 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T08:24:09.086 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T08:24:09.086 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T08:24:09.128 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T08:24:09.165 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T08:24:09.165 INFO:tasks.ceph:config {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T08:24:09.165 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986', 'branch': 'squid', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7695986', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 5498, 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'sleep_before_teardown': 0, 'subset': '27/128', 'suite': 'rbd', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_96805dcf909541c4ca9a1e90258de8ae5b6fa1bc/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc', 'targets': {'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNejBCF/DuKPLpP+eWAWmAkE0gfzD0e90sXSaPzmuiG8JwsuPdxq6g0V8DXPTU4/E1QaKQfHC8Pr2oLlvTh8WjY=', 'smithi149.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJ45Y2WReKifkeHkMYrHECOm6KXf+BP7sUUehrn01mJX4P7Htv8/19aiyf+WmSH2EOVLQ60XiKCOWNuPIQfmr3o='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-07_21:16:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T08:24:09.165 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T08:24:09.196 DEBUG:teuthology.orchestra.run.smithi149:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T08:24:09.212 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T08:24:09.253 DEBUG:teuthology.orchestra.run.smithi149:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T08:24:09.278 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:09.278 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T08:24:09.335 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T08:24:09.335 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-05-08T08:24:09.390 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-05-08 08:24:04.122435205 +0000 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-05-08 08:21:24.447675926 +0000 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-05-08 08:21:24.447675926 +0000 2024-05-08T08:24:09.391 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-05-08 08:21:24.447675926 +0000 2024-05-08T08:24:09.391 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T08:24:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-05-08T08:24:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-05-08T08:24:09.455 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000262255 s, 2.0 MB/s 2024-05-08T08:24:09.457 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T08:24:09.514 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-05-08T08:24:09.569 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-05-08 08:24:04.122435205 +0000 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-05-08 08:21:24.794670591 +0000 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-05-08 08:21:24.794670591 +0000 2024-05-08T08:24:09.570 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-05-08 08:21:24.794670591 +0000 2024-05-08T08:24:09.570 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T08:24:09.634 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-05-08T08:24:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-05-08T08:24:09.635 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000215757 s, 2.4 MB/s 2024-05-08T08:24:09.636 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T08:24:09.693 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-05-08T08:24:09.749 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T08:24:09.749 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:09.749 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T08:24:09.750 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:09.750 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:09.750 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-05-08 08:24:04.123435190 +0000 2024-05-08T08:24:09.750 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-05-08 08:21:25.145665193 +0000 2024-05-08T08:24:09.750 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-05-08 08:21:25.145665193 +0000 2024-05-08T08:24:09.750 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-05-08 08:21:25.145665193 +0000 2024-05-08T08:24:09.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T08:24:09.814 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-05-08T08:24:09.814 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-05-08T08:24:09.814 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000188752 s, 2.7 MB/s 2024-05-08T08:24:09.815 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T08:24:09.873 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-05-08T08:24:09.929 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-05-08 08:24:04.123435190 +0000 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-05-08 08:21:25.494659827 +0000 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-05-08 08:21:25.494659827 +0000 2024-05-08T08:24:09.930 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-05-08 08:21:25.494659827 +0000 2024-05-08T08:24:09.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T08:24:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-05-08T08:24:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-05-08T08:24:09.994 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.00020644 s, 2.5 MB/s 2024-05-08T08:24:09.995 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T08:24:10.053 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:10.053 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:10.053 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T08:24:10.073 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T08:24:10.073 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_1 2024-05-08T08:24:10.135 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-08 08:24:04.003406512 +0000 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-08 08:21:22.522658968 +0000 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-08 08:21:22.522658968 +0000 2024-05-08T08:24:10.136 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-08 08:21:22.522658968 +0000 2024-05-08T08:24:10.136 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T08:24:10.202 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-08T08:24:10.202 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-08T08:24:10.202 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000198301 s, 2.6 MB/s 2024-05-08T08:24:10.204 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T08:24:10.263 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_2 2024-05-08T08:24:10.320 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T08:24:10.320 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:10.320 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-08T08:24:10.320 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:10.320 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:10.321 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-08 08:24:04.003406512 +0000 2024-05-08T08:24:10.321 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-08 08:21:22.873651663 +0000 2024-05-08T08:24:10.321 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-08 08:21:22.873651663 +0000 2024-05-08T08:24:10.321 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-08 08:21:22.873651663 +0000 2024-05-08T08:24:10.321 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T08:24:10.386 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-08T08:24:10.386 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-08T08:24:10.386 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000223367 s, 2.3 MB/s 2024-05-08T08:24:10.387 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T08:24:10.444 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_3 2024-05-08T08:24:10.500 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T08:24:10.500 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:10.501 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-08T08:24:10.501 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:10.501 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:10.501 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-08 08:24:04.003406512 +0000 2024-05-08T08:24:10.501 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-08 08:21:23.210644649 +0000 2024-05-08T08:24:10.501 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-08 08:21:23.210644649 +0000 2024-05-08T08:24:10.501 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-08 08:21:23.210644649 +0000 2024-05-08T08:24:10.501 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T08:24:10.564 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-08T08:24:10.564 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-08T08:24:10.564 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000211716 s, 2.4 MB/s 2024-05-08T08:24:10.565 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T08:24:10.622 DEBUG:teuthology.orchestra.run.smithi149:> stat /dev/vg_nvme/lv_4 2024-05-08T08:24:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T08:24:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T08:24:10.678 INFO:teuthology.orchestra.run.smithi149.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-08T08:24:10.679 INFO:teuthology.orchestra.run.smithi149.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T08:24:10.679 INFO:teuthology.orchestra.run.smithi149.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T08:24:10.679 INFO:teuthology.orchestra.run.smithi149.stdout:Access: 2024-05-08 08:24:04.004406492 +0000 2024-05-08T08:24:10.679 INFO:teuthology.orchestra.run.smithi149.stdout:Modify: 2024-05-08 08:21:23.550637569 +0000 2024-05-08T08:24:10.679 INFO:teuthology.orchestra.run.smithi149.stdout:Change: 2024-05-08 08:21:23.550637569 +0000 2024-05-08T08:24:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: Birth: 2024-05-08 08:21:23.550637569 +0000 2024-05-08T08:24:10.679 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T08:24:10.742 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records in 2024-05-08T08:24:10.743 INFO:teuthology.orchestra.run.smithi149.stderr:1+0 records out 2024-05-08T08:24:10.743 INFO:teuthology.orchestra.run.smithi149.stderr:512 bytes copied, 0.000194447 s, 2.6 MB/s 2024-05-08T08:24:10.744 DEBUG:teuthology.orchestra.run.smithi149:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T08:24:10.801 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:10.801 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi149.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T08:24:10.801 INFO:tasks.ceph:Generating config... 2024-05-08T08:24:10.803 INFO:tasks.ceph:[client] rbd qos iops burst = 100 2024-05-08T08:24:10.803 INFO:tasks.ceph:[client] rbd qos iops limit = 50 2024-05-08T08:24:10.803 INFO:tasks.ceph:[client] rbd qos schedule tick min = 100 2024-05-08T08:24:10.803 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-08T08:24:10.803 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-05-08T08:24:10.804 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-08T08:24:10.804 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T08:24:10.804 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T08:24:10.804 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T08:24:10.804 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T08:24:10.804 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] bluestore compression algorithm = zstd 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T08:24:10.804 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T08:24:10.805 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-08T08:24:10.805 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T08:24:10.806 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T08:24:10.874 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T08:24:10.877 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T08:24:10.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T08:24:10.949 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]'), ('mon.b', '172.21.15.149')] 2024-05-08T08:24:10.950 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.149', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-08T08:24:10.951 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:10.951 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T08:24:11.006 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --add b 172.21.15.149 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: generated fsid 1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:setting min_mon_release = quincy 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 0 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:last_changed 2024-05-08T08:24:11.102916+0000 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-05-08T08:24:11.102916+0000 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:min_mon_release 17 (quincy) 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:election_strategy: 1 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:0: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.a 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:1: [v2:172.21.15.149:3300/0,v1:172.21.15.149:6789/0] mon.b 2024-05-08T08:24:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2024-05-08T08:24:11.096 INFO:teuthology.orchestra.run.smithi031.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T08:24:11.097 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T08:24:11.112 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf... 2024-05-08T08:24:11.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T08:24:11.156 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T08:24:11.197 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-05-08T08:24:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-05-08T08:24:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T08:24:11.197 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T08:24:11.198 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-05-08T08:24:11.199 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-05-08T08:24:11.200 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.149" 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: mon client directed command retry = 5 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on pool no app = False 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: ms inject socket failures = 5000 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-05-08T08:24:11.201 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-05-08T08:24:11.202 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression algorithm = zstd 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression mode = aggressive 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 1/20 2024-05-08T08:24:11.203 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 1/20 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 4/10 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: osd max markdown count = 1000 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: osd shutdown pgref assert = True 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-05-08T08:24:11.204 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-05-08T08:24:11.205 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T08:24:11.206 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: rbd qos iops burst = 100 2024-05-08T08:24:11.207 INFO:teuthology.orchestra.run.smithi031.stdout: rbd qos iops limit = 50 2024-05-08T08:24:11.208 INFO:teuthology.orchestra.run.smithi031.stdout: rbd qos schedule tick min = 100 2024-05-08T08:24:11.208 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-05-08T08:24:11.208 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-05-08T08:24:11.208 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-05-08T08:24:11.208 INFO:teuthology.orchestra.run.smithi149.stdout:[global] 2024-05-08T08:24:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: chdir = "" 2024-05-08T08:24:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: auth supported = cephx 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: filestore xattr use omap = true 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: mon clock drift allowed = 1.000 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: osd crush chooseleaf type = 0 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: auth debug = true 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on old message = true 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: ms die on bug = true 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: mon pg warn max object skew = 0 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default size = 2 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow primary affinity = true 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd allow pg remap = true 2024-05-08T08:24:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on legacy crush tunables = false 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on crush straw calc version zero = false 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on no sortbitwise = false 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on osd down out interval zero = false 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on too few osds = false 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon_allow_pool_size_one = true 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: osd default data pool replay window = 5 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: mon allow pool delete = true 2024-05-08T08:24:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: mon cluster log file level = debug 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: debug asserts on shutdown = true 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: mon health detail to clog = false 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: mon host = "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.149" 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: mon client directed command retry = 5 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: mon warn on pool no app = False 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: ms inject socket failures = 5000 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: fsid = 1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout:[osd] 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: osd journal size = 100 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub load threshold = 5.0 2024-05-08T08:24:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: osd scrub max interval = 600 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd mclock profile = high_recovery_ops 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd recover clone overlap = true 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd recovery max chunk = 1048576 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug shutdown = true 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug op order = true 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug verify stray on activate = true 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug trim objects = true 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: osd open classes on start = true 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug pg log writeout = true 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: osd deep scrub update digest min age = 30 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: osd map max advance = 10 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: journal zero on create = true 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: filestore ondisk finisher threads = 3 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: filestore apply finisher threads = 3 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: bdev debug aio = true 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: osd debug misdirected ops = true 2024-05-08T08:24:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore block size = 96636764160 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore compression algorithm = zstd 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore compression mode = aggressive 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: bluestore fsck on mount = True 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluefs = 1/20 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: debug bluestore = 1/20 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: debug osd = 20 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: debug rocksdb = 4/10 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd full ratio = 0.9 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd nearfull ratio = 0.8 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: osd blocked scrub grace period = 3600 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: osd failsafe full ratio = 0.95 2024-05-08T08:24:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: osd max markdown count = 1000 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: osd objectstore = bluestore 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: osd shutdown pgref assert = True 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr] 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: debug mgr = 20 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-08T08:24:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-08T08:24:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:24:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:24:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: mgr/telemetry/nag = false 2024-05-08T08:24:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.217 INFO:teuthology.orchestra.run.smithi149.stdout:[mon] 2024-05-08T08:24:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: debug ms = 1 2024-05-08T08:24:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: debug mon = 20 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: debug paxos = 20 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: debug auth = 20 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: mon data avail warn = 5 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: mon mgr mkfs grace = 240 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min pgs per osd = 4 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd reporter subtree level = osd 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: mon osd prime pg temp = true 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: mon reweight min bytes per osd = 10 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T08:24:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: # 1m isn't quite enough 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: mon_down_mkfs_grace = 2m 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: mon_warn_on_filestore_osds = false 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout:[client] 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: rgw cache enabled = true 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable ops log = true 2024-05-08T08:24:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: rgw enable usage log = true 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: rbd qos iops burst = 100 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: rbd qos iops limit = 50 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: rbd qos schedule tick min = 100 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.a] 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.c] 2024-05-08T08:24:11.220 INFO:teuthology.orchestra.run.smithi149.stdout:[mon.b] 2024-05-08T08:24:11.222 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T08:24:11.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T08:24:11.317 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T08:24:11.317 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:11.318 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T08:24:11.336 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:11.336 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T08:24:11.393 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:24:11.393 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:11.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T08:24:11.393 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T08:24:11.470 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:11.470 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:11.525 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:24:11.526 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:11.526 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T08:24:11.526 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T08:24:11.562 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:11.562 DEBUG:teuthology.orchestra.run.smithi149:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:11.617 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T08:24:11.618 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T08:24:11.618 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T08:24:11.670 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T08:24:11.673 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T08:24:11.733 INFO:teuthology.orchestra.run.smithi149.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T08:24:11.735 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T08:24:11.736 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T08:24:11.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T08:24:11.778 INFO:teuthology.orchestra.run.smithi031.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T08:24:11.791 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T08:24:11.832 INFO:teuthology.orchestra.run.smithi149.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T08:24:11.845 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T08:24:11.845 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi149.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T08:24:11.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T08:24:11.871 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:11.871 INFO:tasks.ceph:role: osd.0 2024-05-08T08:24:11.871 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:24:11.871 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:11.944 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:12.465 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-05-08T08:24:12.467 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-05-08T08:24:12.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T08:24:12.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T08:24:12.638 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T08:24:12.705 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:12.705 INFO:tasks.ceph:role: osd.1 2024-05-08T08:24:12.705 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:24:12.705 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T08:24:12.779 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:12.780 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:13.239 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-05-08T08:24:13.241 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-05-08T08:24:13.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T08:24:13.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T08:24:13.366 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T08:24:13.432 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:13.432 INFO:tasks.ceph:role: osd.2 2024-05-08T08:24:13.432 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:24:13.432 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:13.505 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:13.506 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:13.506 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:13.990 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-05-08T08:24:13.992 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-05-08T08:24:13.992 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T08:24:14.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T08:24:14.151 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T08:24:14.218 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:14.218 INFO:tasks.ceph:role: osd.3 2024-05-08T08:24:14.218 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:24:14.218 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T08:24:14.291 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:14.291 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:14.292 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:14.830 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-05-08T08:24:14.833 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-05-08T08:24:14.833 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T08:24:14.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T08:24:14.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:15.082 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:15.089+0000 7fed50218640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T08:24:15.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:15.089+0000 7fed50218640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T08:24:15.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:15.089+0000 7fed50218640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T08:24:15.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:15.089+0000 7fed50218640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T08:24:15.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:15.089+0000 7fed50218640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T08:24:15.083 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:15.090+0000 7fed50218640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T08:24:17.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T08:24:17.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:18.071+0000 7fed16c14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T08:24:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:18.072+0000 7fed16c14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T08:24:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:18.072+0000 7fed16c14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T08:24:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:18.072+0000 7fed16c14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T08:24:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:18.072+0000 7fed16c14640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T08:24:18.065 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:18.072+0000 7fed16c14640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T08:24:20.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T08:24:20.951 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:21.043 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:21.050+0000 7fd4a542d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T08:24:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:21.050+0000 7fd4a542d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T08:24:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:21.050+0000 7fd4a542d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T08:24:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:21.050+0000 7fd4a542d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T08:24:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:21.050+0000 7fd4a542d640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T08:24:21.044 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:21.051+0000 7fd4a542d640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T08:24:23.900 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T08:24:23.930 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:24.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:24.030+0000 7fc234d44640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T08:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:24.030+0000 7fc234d44640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T08:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:24.031+0000 7fc234d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T08:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:24.031+0000 7fc234d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T08:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:24.031+0000 7fc234d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T08:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:24:24.031+0000 7fc234d44640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T08:24:26.872 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T08:24:26.902 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T08:24:26.933 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:26.933 INFO:tasks.ceph:role: osd.4 2024-05-08T08:24:26.933 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:24:26.933 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T08:24:27.011 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:27.011 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:27.011 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:27.011 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:27.011 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:27.011 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:27.011 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:27.012 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:27.012 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:27.012 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:27.516 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-08T08:24:27.518 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-08T08:24:27.518 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T08:24:27.634 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T08:24:27.666 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T08:24:27.734 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:27.734 INFO:tasks.ceph:role: osd.5 2024-05-08T08:24:27.734 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:24:27.734 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T08:24:27.807 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:27.807 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:27.807 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:27.807 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:27.807 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:27.807 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:27.808 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:27.808 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:27.808 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:27.808 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:28.265 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-08T08:24:28.268 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-08T08:24:28.268 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T08:24:28.360 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T08:24:28.388 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T08:24:28.453 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:28.453 INFO:tasks.ceph:role: osd.6 2024-05-08T08:24:28.453 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:24:28.453 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:28.525 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:28.526 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:28.526 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:29.021 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-08T08:24:29.023 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-08T08:24:29.023 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T08:24:29.161 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T08:24:29.192 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T08:24:29.258 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T08:24:29.258 INFO:tasks.ceph:role: osd.7 2024-05-08T08:24:29.258 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:24:29.258 DEBUG:teuthology.orchestra.run.smithi149:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T08:24:29.331 INFO:teuthology.orchestra.run.smithi149.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T08:24:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T08:24:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T08:24:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T08:24:29.331 INFO:teuthology.orchestra.run.smithi149.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T08:24:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: = sunit=0 swidth=0 blks 2024-05-08T08:24:29.331 INFO:teuthology.orchestra.run.smithi149.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T08:24:29.332 INFO:teuthology.orchestra.run.smithi149.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T08:24:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T08:24:29.332 INFO:teuthology.orchestra.run.smithi149.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T08:24:29.846 INFO:teuthology.orchestra.run.smithi149.stdout:Discarding blocks...Done. 2024-05-08T08:24:29.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi149.front.sepia.ceph.com -o noatime 2024-05-08T08:24:29.849 DEBUG:teuthology.orchestra.run.smithi149:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T08:24:29.983 DEBUG:teuthology.orchestra.run.smithi149:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T08:24:30.018 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:30.119 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:30.124+0000 7f871d331640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T08:24:30.119 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:30.124+0000 7f871d331640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T08:24:30.119 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:30.125+0000 7f871d331640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T08:24:30.119 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:30.125+0000 7f871d331640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T08:24:30.119 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:30.125+0000 7f871d331640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T08:24:30.119 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:30.125+0000 7f871d331640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T08:24:32.966 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T08:24:32.995 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:33.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:33.091+0000 7f892ec14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T08:24:33.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:33.091+0000 7f892ec14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T08:24:33.086 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:33.092+0000 7f892ec14640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T08:24:33.087 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:33.092+0000 7f892ec14640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T08:24:33.087 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:33.092+0000 7f892ec14640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T08:24:33.087 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:33.092+0000 7f892ec14640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T08:24:35.941 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T08:24:35.969 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:36.057 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:36.063+0000 7f2adb026640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T08:24:36.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:36.064+0000 7f2adb026640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T08:24:36.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:36.064+0000 7f2adb026640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T08:24:36.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:36.064+0000 7f2adb026640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T08:24:36.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:36.064+0000 7f2adb026640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T08:24:36.058 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:36.064+0000 7f2adb026640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T08:24:38.910 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T08:24:38.936 DEBUG:teuthology.orchestra.run.smithi149:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:39.022 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:39.028+0000 7f23669f3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T08:24:39.022 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:39.028+0000 7f23669f3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T08:24:39.022 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:39.028+0000 7f23669f3640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T08:24:39.022 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:39.028+0000 7f23669f3640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T08:24:39.022 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:39.028+0000 7f23669f3640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T08:24:39.023 INFO:teuthology.orchestra.run.smithi149.stderr:2024-05-08T08:24:39.029+0000 7f23669f3640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T08:24:41.866 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T08:24:41.893 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T08:24:41.893 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:41.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T08:24:41.924 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:41.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T08:24:41.995 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:41.995 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T08:24:42.065 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:42.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T08:24:42.135 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:42.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T08:24:42.204 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:42.204 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T08:24:42.232 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:42.232 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T08:24:42.299 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:42.299 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T08:24:42.365 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:42.365 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T08:24:42.431 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:42.431 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T08:24:42.495 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:42.496 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T08:24:42.515 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:42.515 DEBUG:teuthology.orchestra.run.smithi149:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T08:24:42.551 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T08:24:42.551 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T08:24:42.557 DEBUG:teuthology.orchestra.run.smithi149:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCrNjtmuKp2KBAAiFqm4bjKtQFlSYIZUcsoKg== 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCvNjtm/gBlBRAAyIkA5mdbZmBFwzBjqgcOYA== 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCyNjtmyFNVBBAAb45xpdPxUpqawBDMR/Lg7w== 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQC1NjtmFDUSAxAAI3aBSng/4Qh0GT6ku4laLw== 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQC4NjtmofzgARAA4iGqx1z+1dSKmGPOX3PaAg== 2024-05-08T08:24:42.595 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCrNjtmwGEwLBAAoTe7I+jD3T7/Czf3ppNrUQ== 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQC+NjtmnG2HBxAAmXq57PunO4KhroRKtaqYMg== 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDBNjtmhYCQBRAAADvy1/432kcydxAhx00UDg== 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDENjtm+yTnAxAAnNtywzJlt6UjfEnfdgPTVg== 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDHNjtmzxTGARAALKiBCe9iOacPjJiAKCXjng== 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCrNjtmcn/lLhAAcoRiXT7eGBMFg4jfyDAcAw== 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout:[client.1] 2024-05-08T08:24:42.596 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQCrNjtmNJ0dMhAARrTmYmJ6sm07FjQOXr1Cdw== 2024-05-08T08:24:42.614 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr.y] 2024-05-08T08:24:42.614 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCrNjtmuKp2KBAAiFqm4bjKtQFlSYIZUcsoKg== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.0] 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCvNjtm/gBlBRAAyIkA5mdbZmBFwzBjqgcOYA== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.1] 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCyNjtmyFNVBBAAb45xpdPxUpqawBDMR/Lg7w== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.2] 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQC1NjtmFDUSAxAAI3aBSng/4Qh0GT6ku4laLw== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.3] 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQC4NjtmofzgARAA4iGqx1z+1dSKmGPOX3PaAg== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[mgr.x] 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCrNjtmwGEwLBAAoTe7I+jD3T7/Czf3ppNrUQ== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.4] 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQC+NjtmnG2HBxAAmXq57PunO4KhroRKtaqYMg== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.5] 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDBNjtmhYCQBRAAADvy1/432kcydxAhx00UDg== 2024-05-08T08:24:42.615 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.6] 2024-05-08T08:24:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDENjtm+yTnAxAAnNtywzJlt6UjfEnfdgPTVg== 2024-05-08T08:24:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:[osd.7] 2024-05-08T08:24:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQDHNjtmzxTGARAALKiBCe9iOacPjJiAKCXjng== 2024-05-08T08:24:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:[client.0] 2024-05-08T08:24:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCrNjtmcn/lLhAAcoRiXT7eGBMFg4jfyDAcAw== 2024-05-08T08:24:42.616 INFO:teuthology.orchestra.run.smithi149.stdout:[client.1] 2024-05-08T08:24:42.616 INFO:teuthology.orchestra.run.smithi149.stdout: key = AQCrNjtmNJ0dMhAARrTmYmJ6sm07FjQOXr1Cdw== 2024-05-08T08:24:42.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T08:24:42.640 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T08:24:42.708 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.710 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.764 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.766 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.816 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.819 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.869 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.871 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T08:24:42.924 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T08:24:42.974 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:42.976 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:43.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:43.029 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:43.080 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:43.082 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:43.132 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:43.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T08:24:43.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T08:24:43.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T08:24:43.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T08:24:43.243 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T08:24:43.296 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T08:24:43.297 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T08:24:43.322 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T08:24:43.419 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T08:24:43.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T08:24:43.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T08:24:43.606 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T08:24:43.632 DEBUG:teuthology.orchestra.run.smithi149:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T08:24:43.658 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T08:24:43.754 DEBUG:teuthology.orchestra.run.smithi149:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T08:24:43.779 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:43.781 DEBUG:teuthology.orchestra.run.smithi149:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T08:24:43.835 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T08:24:43.835 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T08:24:43.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T08:24:43.837 INFO:tasks.ceph.mon.a:Started 2024-05-08T08:24:43.838 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T08:24:43.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T08:24:43.840 INFO:tasks.ceph.mon.c:Started 2024-05-08T08:24:43.840 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T08:24:43.840 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T08:24:43.878 INFO:tasks.ceph.mon.b:Started 2024-05-08T08:24:43.878 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T08:24:43.878 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T08:24:43.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T08:24:43.881 INFO:tasks.ceph.mgr.y:Started 2024-05-08T08:24:43.881 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T08:24:43.881 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T08:24:43.883 INFO:tasks.ceph.mgr.x:Started 2024-05-08T08:24:43.883 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T08:24:43.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T08:24:44.245 INFO:teuthology.orchestra.run.smithi031.stderr:adjusted tunables profile to default 2024-05-08T08:24:44.256 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T08:24:44.257 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T08:24:44.257 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:44.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T08:24:44.258 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:44.265+0000 7f0689842040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T08:24:44.284 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:44.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T08:24:44.284 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:44.291+0000 7ff4f123c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T08:24:44.350 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:44.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T08:24:44.419 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:44.426+0000 7f0689842040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T08:24:44.421 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-05-08T08:24:44.421 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T08:24:44.472 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:44.479+0000 7f0689842040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T08:24:44.487 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:44.487 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T08:24:44.489 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:44.496+0000 7ff4f123c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T08:24:44.513 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:44.513 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T08:24:44.546 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:44.553+0000 7ff4f123c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T08:24:44.579 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:44.579 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T08:24:44.644 DEBUG:teuthology.orchestra.run.smithi149:> set -ex 2024-05-08T08:24:44.644 DEBUG:teuthology.orchestra.run.smithi149:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T08:24:44.709 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 14afea42-356a-4163-ac3f-2e270e807917 0 2024-05-08T08:24:44.756 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:44.764+0000 7f0689842040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T08:24:44.805 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:44.812+0000 7f0689842040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T08:24:44.825 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:44.832+0000 7ff4f123c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T08:24:44.873 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:44.880+0000 7ff4f123c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T08:24:44.935 INFO:teuthology.orchestra.run.smithi149.stdout:0 2024-05-08T08:24:44.951 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new db77a923-8f49-4b47-b752-00b475917a0b 1 2024-05-08T08:24:45.179 INFO:teuthology.orchestra.run.smithi149.stdout:1 2024-05-08T08:24:45.191 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 3d508691-8e40-4592-82d0-eaf4c4c7d7dd 2 2024-05-08T08:24:45.393 INFO:teuthology.orchestra.run.smithi149.stdout:2 2024-05-08T08:24:45.405 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 2545300a-428a-41ed-b19a-3a0cb9e5e755 3 2024-05-08T08:24:45.582 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:45.589+0000 7f0689842040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T08:24:45.592 INFO:teuthology.orchestra.run.smithi149.stdout:3 2024-05-08T08:24:45.604 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 886aeed5-4495-4064-8579-050d38a4ad1b 4 2024-05-08T08:24:45.640 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:45.647+0000 7ff4f123c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T08:24:45.675 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:45.683+0000 7f0689842040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T08:24:45.720 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:45.728+0000 7f0689842040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T08:24:45.732 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:45.739+0000 7ff4f123c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T08:24:45.777 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:45.784+0000 7ff4f123c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T08:24:45.793 INFO:teuthology.orchestra.run.smithi149.stdout:4 2024-05-08T08:24:45.804 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 4d15cef0-9ee9-4069-a6c4-a650e873ee03 5 2024-05-08T08:24:45.897 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:45.903+0000 7f0689842040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T08:24:45.970 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:45.977+0000 7f0689842040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T08:24:45.970 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:45.978+0000 7ff4f123c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T08:24:46.012 INFO:teuthology.orchestra.run.smithi149.stdout:5 2024-05-08T08:24:46.024 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 910ac664-9573-4851-9e2b-89ac6015fdc1 6 2024-05-08T08:24:46.027 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:46.033+0000 7f0689842040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T08:24:46.028 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:46.035+0000 7ff4f123c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T08:24:46.083 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:46.090+0000 7ff4f123c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T08:24:46.145 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:46.152+0000 7f0689842040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T08:24:46.214 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:46.211+0000 7ff4f123c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T08:24:46.214 INFO:teuthology.orchestra.run.smithi149.stdout:6 2024-05-08T08:24:46.221 DEBUG:teuthology.orchestra.run.smithi149:> sudo ceph --cluster ceph osd new 14cc2d94-48e4-47c9-896a-9be9281ef0c5 7 2024-05-08T08:24:46.429 INFO:teuthology.orchestra.run.smithi149.stdout:7 2024-05-08T08:24:46.440 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T08:24:46.441 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T08:24:46.443 INFO:tasks.ceph.osd.0:Started 2024-05-08T08:24:46.443 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T08:24:46.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T08:24:46.446 INFO:tasks.ceph.osd.1:Started 2024-05-08T08:24:46.446 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T08:24:46.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T08:24:46.448 INFO:tasks.ceph.osd.2:Started 2024-05-08T08:24:46.448 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T08:24:46.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T08:24:46.450 INFO:tasks.ceph.osd.3:Started 2024-05-08T08:24:46.451 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T08:24:46.451 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T08:24:46.453 INFO:tasks.ceph.osd.4:Started 2024-05-08T08:24:46.453 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T08:24:46.453 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T08:24:46.456 INFO:tasks.ceph.osd.5:Started 2024-05-08T08:24:46.456 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T08:24:46.456 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T08:24:46.458 INFO:tasks.ceph.osd.6:Started 2024-05-08T08:24:46.458 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T08:24:46.458 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T08:24:46.460 INFO:tasks.ceph.osd.7:Started 2024-05-08T08:24:46.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:24:46.585 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:46.592+0000 7f0689842040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T08:24:46.646 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:46.654+0000 7ff4f123c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T08:24:46.669 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:46.669 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":10,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:24:46.429925+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T08:24:46.429925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:24:46.680 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T08:24:46.681 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T08:24:46.844 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:46.851+0000 7f0689842040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T08:24:46.895 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:46.903+0000 7f0689842040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T08:24:46.919 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:46.926+0000 7ff4f123c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T08:24:46.974 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:46.981+0000 7ff4f123c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T08:24:47.010 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.017+0000 7f0689842040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T08:24:47.057 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:24:47.064+0000 7fa4c9419640 -1 Falling back to public interface 2024-05-08T08:24:47.057 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:24:47.064+0000 7f8066187640 -1 Falling back to public interface 2024-05-08T08:24:47.058 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:24:47.064+0000 7fa4c7e00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-05-08T08:24:47.058 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:24:47.064+0000 7f8065136640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2024-05-08T08:24:47.058 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:24:47.065+0000 7fa4c7e00640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6802/0: (98) Address already in use 2024-05-08T08:24:47.058 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:24:47.064+0000 7faa85c14640 -1 Falling back to public interface 2024-05-08T08:24:47.066 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:24:47.073+0000 7f84b9a0f640 -1 Falling back to public interface 2024-05-08T08:24:47.073 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:24:47.080+0000 7f890da10640 -1 Falling back to public interface 2024-05-08T08:24:47.085 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:24:47.093+0000 7f3c2941a640 -1 Falling back to public interface 2024-05-08T08:24:47.085 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:24:47.093+0000 7f4c5b80a640 -1 Falling back to public interface 2024-05-08T08:24:47.086 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:24:47.093+0000 7f4c5a7b9640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6816/0: (98) Address already in use 2024-05-08T08:24:47.089 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:24:47.096+0000 7f9029423640 -1 Falling back to public interface 2024-05-08T08:24:47.097 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.105+0000 7ff4f123c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T08:24:47.306 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.313+0000 7f0689842040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T08:24:47.355 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.362+0000 7f0689842040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T08:24:47.408 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.415+0000 7ff4f123c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T08:24:47.458 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.465+0000 7ff4f123c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T08:24:47.486 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T08:24:47.513 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.520+0000 7f0689842040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T08:24:47.626 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.633+0000 7ff4f123c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T08:24:47.651 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.658+0000 7f0689842040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T08:24:47.653 INFO:teuthology.misc.health.smithi031.stdout: 2024-05-08T08:24:47.653 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":10,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:24:46.429925+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T08:24:46.429925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:24:47.663 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T08:24:47.700 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.707+0000 7f0689842040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T08:24:47.769 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.776+0000 7f0689842040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T08:24:47.772 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.779+0000 7ff4f123c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T08:24:47.814 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:47.821+0000 7f0689842040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T08:24:47.824 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.831+0000 7ff4f123c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T08:24:47.898 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.905+0000 7ff4f123c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T08:24:47.946 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:47.953+0000 7ff4f123c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T08:24:48.289 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:48.297+0000 7f0689842040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T08:24:48.404 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:48.411+0000 7f0689842040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T08:24:48.448 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:48.455+0000 7ff4f123c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T08:24:48.569 INFO:tasks.ceph.mgr.y.smithi031.stderr:2024-05-08T08:24:48.576+0000 7ff4f123c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T08:24:49.418 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:49.426+0000 7f062a514640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T08:24:49.419 INFO:tasks.ceph.mgr.x.smithi149.stderr:2024-05-08T08:24:49.426+0000 7f062a514640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T08:24:50.534 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:24:50.541+0000 7f8066187640 -1 osd.6 0 log_to_monitors true 2024-05-08T08:24:50.572 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:24:50.579+0000 7faa85c14640 -1 osd.0 0 log_to_monitors true 2024-05-08T08:24:50.579 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:24:50.586+0000 7fa4c9419640 -1 osd.4 0 log_to_monitors true 2024-05-08T08:24:50.581 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:24:50.589+0000 7f84b9a0f640 -1 osd.2 0 log_to_monitors true 2024-05-08T08:24:50.584 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:24:50.591+0000 7f890da10640 -1 osd.1 0 log_to_monitors true 2024-05-08T08:24:50.585 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:24:50.593+0000 7f3c2941a640 -1 osd.5 0 log_to_monitors true 2024-05-08T08:24:50.589 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:24:50.596+0000 7f9029423640 -1 osd.3 0 log_to_monitors true 2024-05-08T08:24:50.608 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:24:50.615+0000 7f4c5b80a640 -1 osd.7 0 log_to_monitors true 2024-05-08T08:24:53.526 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:24:53.533+0000 7f3c24e7d640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T08:24:53.535 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:24:53.543+0000 7fa4c4de7640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T08:24:53.541 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:24:53.549+0000 7f3c1cc9d640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:53.549 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:24:53.557+0000 7fa4bcc07640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:53.557 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:24:53.565+0000 7f4c577a0640 -1 osd.7 0 waiting for initial osdmap 2024-05-08T08:24:53.558 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:24:53.566+0000 7f806211d640 -1 osd.6 0 waiting for initial osdmap 2024-05-08T08:24:53.573 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:24:53.581+0000 7f8059f3d640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:53.574 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:24:53.581+0000 7f4c4f5c0640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:53.657 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:24:53.664+0000 7faa8155c640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T08:24:53.663 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:24:53.670+0000 7f8909451640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T08:24:53.670 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:24:53.678+0000 7faa7937c640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:53.671 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:24:53.678+0000 7f84b5422640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T08:24:53.673 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:24:53.681+0000 7f8901271640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:53.676 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:24:53.683+0000 7f9024d6a640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T08:24:53.682 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:24:53.690+0000 7f84ad242640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:53.694 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:24:53.702+0000 7f901cb8a640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:24:54.471 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T08:24:54.783 INFO:teuthology.misc.health.smithi031.stdout: 2024-05-08T08:24:54.783 INFO:teuthology.misc.health.smithi031.stdout:{"epoch":13,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:24:54.547557+0000","last_up_change":"2024-05-08T08:24:54.547557+0000","last_in_change":"2024-05-08T08:24:46.429925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6801","nonce":727789107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6803","nonce":727789107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6807","nonce":727789107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6805","nonce":727789107}]},"public_addr":"172.21.15.31:6801/727789107","cluster_addr":"172.21.15.31:6803/727789107","heartbeat_back_addr":"172.21.15.31:6807/727789107","heartbeat_front_addr":"172.21.15.31:6805/727789107","state":["exists","up"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6817","nonce":3462314176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6819","nonce":3462314176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6823","nonce":3462314176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6821","nonce":3462314176}]},"public_addr":"172.21.15.31:6817/3462314176","cluster_addr":"172.21.15.31:6819/3462314176","heartbeat_back_addr":"172.21.15.31:6823/3462314176","heartbeat_front_addr":"172.21.15.31:6821/3462314176","state":["exists","up"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6809","nonce":2736372115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6811","nonce":2736372115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6815","nonce":2736372115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6813","nonce":2736372115}]},"public_addr":"172.21.15.31:6809/2736372115","cluster_addr":"172.21.15.31:6811/2736372115","heartbeat_back_addr":"172.21.15.31:6815/2736372115","heartbeat_front_addr":"172.21.15.31:6813/2736372115","state":["exists","up"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6825","nonce":573482428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6827","nonce":573482428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6831","nonce":573482428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6829","nonce":573482428}]},"public_addr":"172.21.15.31:6825/573482428","cluster_addr":"172.21.15.31:6827/573482428","heartbeat_back_addr":"172.21.15.31:6831/573482428","heartbeat_front_addr":"172.21.15.31:6829/573482428","state":["exists","up"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6803","nonce":3424463091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6807","nonce":3424463091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6815","nonce":3424463091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6811","nonce":3424463091}]},"public_addr":"172.21.15.149:6803/3424463091","cluster_addr":"172.21.15.149:6807/3424463091","heartbeat_back_addr":"172.21.15.149:6815/3424463091","heartbeat_front_addr":"172.21.15.149:6811/3424463091","state":["exists","up"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6818","nonce":3709461555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6822","nonce":3709461555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6830","nonce":3709461555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6826","nonce":3709461555}]},"public_addr":"172.21.15.149:6818/3709461555","cluster_addr":"172.21.15.149:6822/3709461555","heartbeat_back_addr":"172.21.15.149:6830/3709461555","heartbeat_front_addr":"172.21.15.149:6826/3709461555","state":["exists","up"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6802","nonce":405561966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6806","nonce":405561966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6814","nonce":405561966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6810","nonce":405561966}]},"public_addr":"172.21.15.149:6802/405561966","cluster_addr":"172.21.15.149:6806/405561966","heartbeat_back_addr":"172.21.15.149:6814/405561966","heartbeat_front_addr":"172.21.15.149:6810/405561966","state":["exists","up"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6817","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6819","nonce":570236217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6821","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6823","nonce":570236217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6829","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6831","nonce":570236217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6825","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6827","nonce":570236217}]},"public_addr":"172.21.15.149:6819/570236217","cluster_addr":"172.21.15.149:6823/570236217","heartbeat_back_addr":"172.21.15.149:6831/570236217","heartbeat_front_addr":"172.21.15.149:6827/570236217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:24:54.795 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T08:24:54.795 INFO:tasks.ceph:Creating RBD pool 2024-05-08T08:24:54.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-08T08:24:55.542 INFO:teuthology.orchestra.run.smithi031.stderr:pool 'rbd' created 2024-05-08T08:24:55.558 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-08T08:24:56.548 INFO:teuthology.orchestra.run.smithi031.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-08T08:24:56.567 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T08:24:56.567 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T08:24:56.567 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T08:24:56.567 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T08:24:56.568 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T08:24:56.931 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:56.941 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":4,"flags":0,"active_gid":4108,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6832","nonce":2107776689},{"type":"v1","addr":"172.21.15.149:6833","nonce":2107776689}]},"active_addr":"172.21.15.149:6833/2107776689","active_change":"2024-05-08T08:24:48.414440+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.149:0","nonce":3425589641}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.149:0","nonce":1579735437}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.149:0","nonce":2934441766}]}]} 2024-05-08T08:24:56.948 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T08:24:56.948 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T08:24:56.948 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:24:57.250 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:57.250 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":15,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:24:56.551148+0000","last_up_change":"2024-05-08T08:24:54.547557+0000","last_in_change":"2024-05-08T08:24:46.429925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T08:24:55.134472+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T08:24:56.465515+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6801","nonce":727789107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6803","nonce":727789107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6807","nonce":727789107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6805","nonce":727789107}]},"public_addr":"172.21.15.31:6801/727789107","cluster_addr":"172.21.15.31:6803/727789107","heartbeat_back_addr":"172.21.15.31:6807/727789107","heartbeat_front_addr":"172.21.15.31:6805/727789107","state":["exists","up"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6817","nonce":3462314176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6819","nonce":3462314176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6823","nonce":3462314176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6821","nonce":3462314176}]},"public_addr":"172.21.15.31:6817/3462314176","cluster_addr":"172.21.15.31:6819/3462314176","heartbeat_back_addr":"172.21.15.31:6823/3462314176","heartbeat_front_addr":"172.21.15.31:6821/3462314176","state":["exists","up"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6809","nonce":2736372115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6811","nonce":2736372115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6815","nonce":2736372115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6813","nonce":2736372115}]},"public_addr":"172.21.15.31:6809/2736372115","cluster_addr":"172.21.15.31:6811/2736372115","heartbeat_back_addr":"172.21.15.31:6815/2736372115","heartbeat_front_addr":"172.21.15.31:6813/2736372115","state":["exists","up"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6825","nonce":573482428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6827","nonce":573482428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6831","nonce":573482428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6829","nonce":573482428}]},"public_addr":"172.21.15.31:6825/573482428","cluster_addr":"172.21.15.31:6827/573482428","heartbeat_back_addr":"172.21.15.31:6831/573482428","heartbeat_front_addr":"172.21.15.31:6829/573482428","state":["exists","up"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6803","nonce":3424463091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6807","nonce":3424463091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6815","nonce":3424463091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6811","nonce":3424463091}]},"public_addr":"172.21.15.149:6803/3424463091","cluster_addr":"172.21.15.149:6807/3424463091","heartbeat_back_addr":"172.21.15.149:6815/3424463091","heartbeat_front_addr":"172.21.15.149:6811/3424463091","state":["exists","up"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6818","nonce":3709461555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6822","nonce":3709461555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6830","nonce":3709461555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6826","nonce":3709461555}]},"public_addr":"172.21.15.149:6818/3709461555","cluster_addr":"172.21.15.149:6822/3709461555","heartbeat_back_addr":"172.21.15.149:6830/3709461555","heartbeat_front_addr":"172.21.15.149:6826/3709461555","state":["exists","up"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6802","nonce":405561966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6806","nonce":405561966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6814","nonce":405561966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6810","nonce":405561966}]},"public_addr":"172.21.15.149:6802/405561966","cluster_addr":"172.21.15.149:6806/405561966","heartbeat_back_addr":"172.21.15.149:6814/405561966","heartbeat_front_addr":"172.21.15.149:6810/405561966","state":["exists","up"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6817","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6819","nonce":570236217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6821","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6823","nonce":570236217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6829","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6831","nonce":570236217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6825","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6827","nonce":570236217}]},"public_addr":"172.21.15.149:6819/570236217","cluster_addr":"172.21.15.149:6823/570236217","heartbeat_back_addr":"172.21.15.149:6831/570236217","heartbeat_front_addr":"172.21.15.149:6827/570236217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.572981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.626543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.638878+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.599293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.565676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.552124+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.531428+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.658391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:24:57.262 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T08:24:57.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:24:57.564 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:24:57.564 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":16,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:24:57.556287+0000","last_up_change":"2024-05-08T08:24:54.547557+0000","last_in_change":"2024-05-08T08:24:46.429925+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T08:24:55.134472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T08:24:56.465515+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6801","nonce":727789107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6803","nonce":727789107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6807","nonce":727789107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6805","nonce":727789107}]},"public_addr":"172.21.15.31:6801/727789107","cluster_addr":"172.21.15.31:6803/727789107","heartbeat_back_addr":"172.21.15.31:6807/727789107","heartbeat_front_addr":"172.21.15.31:6805/727789107","state":["exists","up"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6817","nonce":3462314176}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6819","nonce":3462314176}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6823","nonce":3462314176}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3462314176},{"type":"v1","addr":"172.21.15.31:6821","nonce":3462314176}]},"public_addr":"172.21.15.31:6817/3462314176","cluster_addr":"172.21.15.31:6819/3462314176","heartbeat_back_addr":"172.21.15.31:6823/3462314176","heartbeat_front_addr":"172.21.15.31:6821/3462314176","state":["exists","up"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6809","nonce":2736372115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6811","nonce":2736372115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6815","nonce":2736372115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6813","nonce":2736372115}]},"public_addr":"172.21.15.31:6809/2736372115","cluster_addr":"172.21.15.31:6811/2736372115","heartbeat_back_addr":"172.21.15.31:6815/2736372115","heartbeat_front_addr":"172.21.15.31:6813/2736372115","state":["exists","up"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6825","nonce":573482428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6827","nonce":573482428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6831","nonce":573482428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6829","nonce":573482428}]},"public_addr":"172.21.15.31:6825/573482428","cluster_addr":"172.21.15.31:6827/573482428","heartbeat_back_addr":"172.21.15.31:6831/573482428","heartbeat_front_addr":"172.21.15.31:6829/573482428","state":["exists","up"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6803","nonce":3424463091}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6807","nonce":3424463091}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6815","nonce":3424463091}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":3424463091},{"type":"v1","addr":"172.21.15.149:6811","nonce":3424463091}]},"public_addr":"172.21.15.149:6803/3424463091","cluster_addr":"172.21.15.149:6807/3424463091","heartbeat_back_addr":"172.21.15.149:6815/3424463091","heartbeat_front_addr":"172.21.15.149:6811/3424463091","state":["exists","up"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6818","nonce":3709461555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6822","nonce":3709461555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6830","nonce":3709461555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6826","nonce":3709461555}]},"public_addr":"172.21.15.149:6818/3709461555","cluster_addr":"172.21.15.149:6822/3709461555","heartbeat_back_addr":"172.21.15.149:6830/3709461555","heartbeat_front_addr":"172.21.15.149:6826/3709461555","state":["exists","up"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6802","nonce":405561966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6806","nonce":405561966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6814","nonce":405561966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6810","nonce":405561966}]},"public_addr":"172.21.15.149:6802/405561966","cluster_addr":"172.21.15.149:6806/405561966","heartbeat_back_addr":"172.21.15.149:6814/405561966","heartbeat_front_addr":"172.21.15.149:6810/405561966","state":["exists","up"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6817","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6819","nonce":570236217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6821","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6823","nonce":570236217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6829","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6831","nonce":570236217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6825","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6827","nonce":570236217}]},"public_addr":"172.21.15.149:6819/570236217","cluster_addr":"172.21.15.149:6823/570236217","heartbeat_back_addr":"172.21.15.149:6831/570236217","heartbeat_front_addr":"172.21.15.149:6827/570236217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.572981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.626543+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.638878+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.599293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.565676+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.552124+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.531428+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.658391+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:24:57.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T08:24:57.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T08:24:57.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T08:24:57.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T08:24:57.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T08:24:57.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T08:24:57.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T08:24:57.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T08:24:57.838 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:57.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T08:24:57.842 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:57.842 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:24:57.848 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:57.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:24:57.853 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:57.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T08:24:57.855 INFO:teuthology.orchestra.run.smithi031.stdout:55834574852 2024-05-08T08:24:57.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:24:57.866 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:57.866 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:24:57.867 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:57.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:24:58.003 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:58.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:24:58.479 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-05-08T08:24:58.496 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-05-08T08:24:58.504 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-05-08T08:24:58.506 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-05-08T08:24:58.512 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-05-08T08:24:58.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-05-08T08:24:58.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.2 2024-05-08T08:24:58.526 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:24:58.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-05-08T08:24:58.545 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.3 2024-05-08T08:24:58.577 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-05-08T08:24:58.589 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-05-08T08:24:58.595 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.1 2024-05-08T08:24:58.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-05-08T08:24:58.616 INFO:teuthology.orchestra.run.smithi031.stdout:55834574850 2024-05-08T08:24:58.630 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-05-08T08:24:59.505 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:24:59.517 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T08:24:59.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:24:59.534 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T08:24:59.546 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:24:59.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:24:59.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:24:59.631 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:25:00.099 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:25:00.124 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-05-08T08:25:00.125 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.130 INFO:teuthology.orchestra.run.smithi031.stdout:55834574852 2024-05-08T08:25:00.146 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:25:00.151 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.3 2024-05-08T08:25:00.151 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.167 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.2 2024-05-08T08:25:00.167 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:25:00.167 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.174 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:25:00.187 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-05-08T08:25:00.187 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.188 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.7 2024-05-08T08:25:00.188 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.197 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:25:00.200 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:25:00.209 INFO:teuthology.orchestra.run.smithi031.stdout:55834574851 2024-05-08T08:25:00.211 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.1 2024-05-08T08:25:00.211 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.215 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2024-05-08T08:25:00.215 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.224 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.0 2024-05-08T08:25:00.224 DEBUG:teuthology.parallel:result is None 2024-05-08T08:25:00.224 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T08:25:00.224 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:00.560 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:00.561 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:00.574 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T08:25:00.421719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217336,"kb_used_data":1156,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757384,"statfs":{"total":773094113280,"available":772871561216,"internally_reserved":0,"allocated":1183744,"data_stored":346530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.565157+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:24:56.565157+0000","last_peered":"2024-05-08T08:24:56.565157+0000","last_clean":"2024-05-08T08:24:56.565157+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:24:56.565157+0000","last_undegraded":"2024-05-08T08:24:56.565157+0000","last_fullsized":"2024-05-08T08:24:56.565157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:08.848081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.681452+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:24:56.681452+0000","last_peered":"2024-05-08T08:24:56.681452+0000","last_clean":"2024-05-08T08:24:56.681452+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:24:56.681452+0000","last_undegraded":"2024-05-08T08:24:56.681452+0000","last_fullsized":"2024-05-08T08:24:56.681452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:09.742176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.563632+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:24:56.563632+0000","last_peered":"2024-05-08T08:24:56.563632+0000","last_clean":"2024-05-08T08:24:56.563632+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:24:56.563632+0000","last_undegraded":"2024-05-08T08:24:56.563632+0000","last_fullsized":"2024-05-08T08:24:56.563632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:49.354969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.566858+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:24:56.566858+0000","last_peered":"2024-05-08T08:24:56.566858+0000","last_clean":"2024-05-08T08:24:56.566858+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:24:56.566858+0000","last_undegraded":"2024-05-08T08:24:56.566858+0000","last_fullsized":"2024-05-08T08:24:56.566858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:45.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'64","reported_seq":97,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.750137+0000","last_change":"2024-05-08T08:24:57.620702+0000","last_active":"2024-05-08T08:24:57.750137+0000","last_peered":"2024-05-08T08:24:57.750137+0000","last_clean":"2024-05-08T08:24:57.750137+0000","last_became_active":"2024-05-08T08:24:57.620315+0000","last_became_peered":"2024-05-08T08:24:57.620315+0000","last_unstale":"2024-05-08T08:24:57.750137+0000","last_undegraded":"2024-05-08T08:24:57.750137+0000","last_fullsized":"2024-05-08T08:24:57.750137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:26.416641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.618820+0000","last_change":"2024-05-08T08:24:56.565129+0000","last_active":"2024-05-08T08:24:57.618820+0000","last_peered":"2024-05-08T08:24:57.618820+0000","last_clean":"2024-05-08T08:24:57.618820+0000","last_became_active":"2024-05-08T08:24:56.564785+0000","last_became_peered":"2024-05-08T08:24:56.564785+0000","last_unstale":"2024-05-08T08:24:57.618820+0000","last_undegraded":"2024-05-08T08:24:57.618820+0000","last_fullsized":"2024-05-08T08:24:57.618820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:36:03.832696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.559976+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:24:57.559976+0000","last_peered":"2024-05-08T08:24:57.559976+0000","last_clean":"2024-05-08T08:24:57.559976+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:24:57.559976+0000","last_undegraded":"2024-05-08T08:24:57.559976+0000","last_fullsized":"2024-05-08T08:24:57.559976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":918560,"data_compressed":1376,"data_compressed_allocated":57344,"data_compressed_original":901120,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:00.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:00.876 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:00.876 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:00.889 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T08:25:00.421719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217336,"kb_used_data":1156,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757384,"statfs":{"total":773094113280,"available":772871561216,"internally_reserved":0,"allocated":1183744,"data_stored":346530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.565157+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:24:56.565157+0000","last_peered":"2024-05-08T08:24:56.565157+0000","last_clean":"2024-05-08T08:24:56.565157+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:24:56.565157+0000","last_undegraded":"2024-05-08T08:24:56.565157+0000","last_fullsized":"2024-05-08T08:24:56.565157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:08.848081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.681452+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:24:56.681452+0000","last_peered":"2024-05-08T08:24:56.681452+0000","last_clean":"2024-05-08T08:24:56.681452+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:24:56.681452+0000","last_undegraded":"2024-05-08T08:24:56.681452+0000","last_fullsized":"2024-05-08T08:24:56.681452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:09.742176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.563632+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:24:56.563632+0000","last_peered":"2024-05-08T08:24:56.563632+0000","last_clean":"2024-05-08T08:24:56.563632+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:24:56.563632+0000","last_undegraded":"2024-05-08T08:24:56.563632+0000","last_fullsized":"2024-05-08T08:24:56.563632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:49.354969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.566858+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:24:56.566858+0000","last_peered":"2024-05-08T08:24:56.566858+0000","last_clean":"2024-05-08T08:24:56.566858+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:24:56.566858+0000","last_undegraded":"2024-05-08T08:24:56.566858+0000","last_fullsized":"2024-05-08T08:24:56.566858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:45.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'64","reported_seq":97,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.750137+0000","last_change":"2024-05-08T08:24:57.620702+0000","last_active":"2024-05-08T08:24:57.750137+0000","last_peered":"2024-05-08T08:24:57.750137+0000","last_clean":"2024-05-08T08:24:57.750137+0000","last_became_active":"2024-05-08T08:24:57.620315+0000","last_became_peered":"2024-05-08T08:24:57.620315+0000","last_unstale":"2024-05-08T08:24:57.750137+0000","last_undegraded":"2024-05-08T08:24:57.750137+0000","last_fullsized":"2024-05-08T08:24:57.750137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:26.416641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.618820+0000","last_change":"2024-05-08T08:24:56.565129+0000","last_active":"2024-05-08T08:24:57.618820+0000","last_peered":"2024-05-08T08:24:57.618820+0000","last_clean":"2024-05-08T08:24:57.618820+0000","last_became_active":"2024-05-08T08:24:56.564785+0000","last_became_peered":"2024-05-08T08:24:56.564785+0000","last_unstale":"2024-05-08T08:24:57.618820+0000","last_undegraded":"2024-05-08T08:24:57.618820+0000","last_fullsized":"2024-05-08T08:24:57.618820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:36:03.832696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.559976+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:24:57.559976+0000","last_peered":"2024-05-08T08:24:57.559976+0000","last_clean":"2024-05-08T08:24:57.559976+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:24:57.559976+0000","last_undegraded":"2024-05-08T08:24:57.559976+0000","last_fullsized":"2024-05-08T08:24:57.559976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":918560,"data_compressed":1376,"data_compressed_allocated":57344,"data_compressed_original":901120,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:00.889 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T08:25:00.890 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T08:25:00.890 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T08:25:00.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T08:25:01.227 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:01.227 INFO:teuthology.orchestra.run.smithi031.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T08:25:01.240 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T08:25:01.240 INFO:teuthology.run_tasks:Running task thrashosds... 2024-05-08T08:25:01.254 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-05-08T08:25:01.254 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'rbd qos iops burst': 100, 'rbd qos iops limit': 50, 'rbd qos schedule tick min': 100}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'zstd', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': '96805dcf909541c4ca9a1e90258de8ae5b6fa1bc'}} 2024-05-08T08:25:01.254 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-05-08T08:25:01.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-05-08T08:25:01.572 INFO:tasks.thrashosds:Beginning thrashosds... 2024-05-08T08:25:01.572 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T08:25:01.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:01.868 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:01.869 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:01.880 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T08:25:00.421719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217336,"kb_used_data":1156,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757384,"statfs":{"total":773094113280,"available":772871561216,"internally_reserved":0,"allocated":1183744,"data_stored":346530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.565157+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:24:56.565157+0000","last_peered":"2024-05-08T08:24:56.565157+0000","last_clean":"2024-05-08T08:24:56.565157+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:24:56.565157+0000","last_undegraded":"2024-05-08T08:24:56.565157+0000","last_fullsized":"2024-05-08T08:24:56.565157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:08.848081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.681452+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:24:56.681452+0000","last_peered":"2024-05-08T08:24:56.681452+0000","last_clean":"2024-05-08T08:24:56.681452+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:24:56.681452+0000","last_undegraded":"2024-05-08T08:24:56.681452+0000","last_fullsized":"2024-05-08T08:24:56.681452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:09.742176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.563632+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:24:56.563632+0000","last_peered":"2024-05-08T08:24:56.563632+0000","last_clean":"2024-05-08T08:24:56.563632+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:24:56.563632+0000","last_undegraded":"2024-05-08T08:24:56.563632+0000","last_fullsized":"2024-05-08T08:24:56.563632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:49.354969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.566858+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:24:56.566858+0000","last_peered":"2024-05-08T08:24:56.566858+0000","last_clean":"2024-05-08T08:24:56.566858+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:24:56.566858+0000","last_undegraded":"2024-05-08T08:24:56.566858+0000","last_fullsized":"2024-05-08T08:24:56.566858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:45.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'64","reported_seq":97,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.750137+0000","last_change":"2024-05-08T08:24:57.620702+0000","last_active":"2024-05-08T08:24:57.750137+0000","last_peered":"2024-05-08T08:24:57.750137+0000","last_clean":"2024-05-08T08:24:57.750137+0000","last_became_active":"2024-05-08T08:24:57.620315+0000","last_became_peered":"2024-05-08T08:24:57.620315+0000","last_unstale":"2024-05-08T08:24:57.750137+0000","last_undegraded":"2024-05-08T08:24:57.750137+0000","last_fullsized":"2024-05-08T08:24:57.750137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:26.416641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.618820+0000","last_change":"2024-05-08T08:24:56.565129+0000","last_active":"2024-05-08T08:24:57.618820+0000","last_peered":"2024-05-08T08:24:57.618820+0000","last_clean":"2024-05-08T08:24:57.618820+0000","last_became_active":"2024-05-08T08:24:56.564785+0000","last_became_peered":"2024-05-08T08:24:56.564785+0000","last_unstale":"2024-05-08T08:24:57.618820+0000","last_undegraded":"2024-05-08T08:24:57.618820+0000","last_fullsized":"2024-05-08T08:24:57.618820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:36:03.832696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.559976+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:24:57.559976+0000","last_peered":"2024-05-08T08:24:57.559976+0000","last_clean":"2024-05-08T08:24:57.559976+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:24:57.559976+0000","last_undegraded":"2024-05-08T08:24:57.559976+0000","last_fullsized":"2024-05-08T08:24:57.559976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":918560,"data_compressed":1376,"data_compressed_allocated":57344,"data_compressed_original":901120,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:01.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:02.175 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:02.187 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T08:25:00.421719+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":10,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":217336,"kb_used_data":1156,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754757384,"statfs":{"total":773094113280,"available":772871561216,"internally_reserved":0,"allocated":1183744,"data_stored":346530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12749,"internal_metadata":221236787},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000390"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.565157+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:24:56.565157+0000","last_peered":"2024-05-08T08:24:56.565157+0000","last_clean":"2024-05-08T08:24:56.565157+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:24:56.565157+0000","last_undegraded":"2024-05-08T08:24:56.565157+0000","last_fullsized":"2024-05-08T08:24:56.565157+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:20:08.848081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.681452+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:24:56.681452+0000","last_peered":"2024-05-08T08:24:56.681452+0000","last_clean":"2024-05-08T08:24:56.681452+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:24:56.681452+0000","last_undegraded":"2024-05-08T08:24:56.681452+0000","last_fullsized":"2024-05-08T08:24:56.681452+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:59:09.742176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.563632+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:24:56.563632+0000","last_peered":"2024-05-08T08:24:56.563632+0000","last_clean":"2024-05-08T08:24:56.563632+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:24:56.563632+0000","last_undegraded":"2024-05-08T08:24:56.563632+0000","last_fullsized":"2024-05-08T08:24:56.563632+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:32:49.354969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-05-08T08:24:56.566858+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:24:56.566858+0000","last_peered":"2024-05-08T08:24:56.566858+0000","last_clean":"2024-05-08T08:24:56.566858+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:24:56.566858+0000","last_undegraded":"2024-05-08T08:24:56.566858+0000","last_fullsized":"2024-05-08T08:24:56.566858+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:45.572881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"16'64","reported_seq":97,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.750137+0000","last_change":"2024-05-08T08:24:57.620702+0000","last_active":"2024-05-08T08:24:57.750137+0000","last_peered":"2024-05-08T08:24:57.750137+0000","last_clean":"2024-05-08T08:24:57.750137+0000","last_became_active":"2024-05-08T08:24:57.620315+0000","last_became_peered":"2024-05-08T08:24:57.620315+0000","last_unstale":"2024-05-08T08:24:57.750137+0000","last_undegraded":"2024-05-08T08:24:57.750137+0000","last_fullsized":"2024-05-08T08:24:57.750137+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:25:26.416641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.618820+0000","last_change":"2024-05-08T08:24:56.565129+0000","last_active":"2024-05-08T08:24:57.618820+0000","last_peered":"2024-05-08T08:24:57.618820+0000","last_clean":"2024-05-08T08:24:57.618820+0000","last_became_active":"2024-05-08T08:24:56.564785+0000","last_became_peered":"2024-05-08T08:24:56.564785+0000","last_unstale":"2024-05-08T08:24:57.618820+0000","last_undegraded":"2024-05-08T08:24:57.618820+0000","last_fullsized":"2024-05-08T08:24:57.618820+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:36:03.832696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.559976+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:24:57.559976+0000","last_peered":"2024-05-08T08:24:57.559976+0000","last_clean":"2024-05-08T08:24:57.559976+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:24:57.559976+0000","last_undegraded":"2024-05-08T08:24:57.559976+0000","last_fullsized":"2024-05-08T08:24:57.559976+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T08:24:57.560237+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:24:57.560237+0000","last_peered":"2024-05-08T08:24:57.560237+0000","last_clean":"2024-05-08T08:24:57.560237+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:24:57.560237+0000","last_undegraded":"2024-05-08T08:24:57.560237+0000","last_fullsized":"2024-05-08T08:24:57.560237+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":86,"num_read_kb":73,"num_write":99,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":918560,"data_compressed":1376,"data_compressed_allocated":57344,"data_compressed_original":901120,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":140,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":143360,"data_stored":41019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":155648,"data_stored":47145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":459280,"data_compressed":688,"data_compressed_allocated":28672,"data_compressed_original":450560,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:02.188 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T08:25:02.188 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-05-08T08:25:02.489 INFO:teuthology.orchestra.run.smithi031.stdout:epoch 17 2024-05-08T08:25:02.489 INFO:teuthology.orchestra.run.smithi031.stdout:fsid 1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf 2024-05-08T08:25:02.489 INFO:teuthology.orchestra.run.smithi031.stdout:created 2024-05-08T08:24:44.143941+0000 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:modified 2024-05-08T08:24:58.562710+0000 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:crush_version 4 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:full_ratio 0.95 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:backfillfull_ratio 0.9 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:nearfull_ratio 0.85 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:require_min_compat_client luminous 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:min_compat_client jewel 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:require_osd_release squid 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:stretch_mode_enabled false 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 application rbd read_balance_score 2.00 2024-05-08T08:25:02.490 INFO:teuthology.orchestra.run.smithi031.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:max_osd 8 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/727789107,v1:172.21.15.31:6801/727789107] [v2:172.21.15.31:6802/727789107,v1:172.21.15.31:6803/727789107] exists,up 14afea42-356a-4163-ac3f-2e270e807917 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/3462314176,v1:172.21.15.31:6817/3462314176] [v2:172.21.15.31:6818/3462314176,v1:172.21.15.31:6819/3462314176] exists,up db77a923-8f49-4b47-b752-00b475917a0b 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/2736372115,v1:172.21.15.31:6809/2736372115] [v2:172.21.15.31:6810/2736372115,v1:172.21.15.31:6811/2736372115] exists,up 3d508691-8e40-4592-82d0-eaf4c4c7d7dd 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6824/573482428,v1:172.21.15.31:6825/573482428] [v2:172.21.15.31:6826/573482428,v1:172.21.15.31:6827/573482428] exists,up 2545300a-428a-41ed-b19a-3a0cb9e5e755 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6801/3424463091,v1:172.21.15.149:6803/3424463091] [v2:172.21.15.149:6805/3424463091,v1:172.21.15.149:6807/3424463091] exists,up 886aeed5-4495-4064-8579-050d38a4ad1b 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/3709461555,v1:172.21.15.149:6818/3709461555] [v2:172.21.15.149:6820/3709461555,v1:172.21.15.149:6822/3709461555] exists,up 4d15cef0-9ee9-4069-a6c4-a650e873ee03 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/405561966,v1:172.21.15.149:6802/405561966] [v2:172.21.15.149:6804/405561966,v1:172.21.15.149:6806/405561966] exists,up 910ac664-9573-4851-9e2b-89ac6015fdc1 2024-05-08T08:25:02.491 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6817/570236217,v1:172.21.15.149:6819/570236217] [v2:172.21.15.149:6821/570236217,v1:172.21.15.149:6823/570236217] exists,up 14cc2d94-48e4-47c9-896a-9be9281ef0c5 2024-05-08T08:25:02.502 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6800/727789107,v1:172.21.15.31:6801/727789107] [v2:172.21.15.31:6802/727789107,v1:172.21.15.31:6803/727789107] exists,up 14afea42-356a-4163-ac3f-2e270e807917', 'osd.1 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6816/3462314176,v1:172.21.15.31:6817/3462314176] [v2:172.21.15.31:6818/3462314176,v1:172.21.15.31:6819/3462314176] exists,up db77a923-8f49-4b47-b752-00b475917a0b', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6808/2736372115,v1:172.21.15.31:6809/2736372115] [v2:172.21.15.31:6810/2736372115,v1:172.21.15.31:6811/2736372115] exists,up 3d508691-8e40-4592-82d0-eaf4c4c7d7dd', 'osd.3 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.31:6824/573482428,v1:172.21.15.31:6825/573482428] [v2:172.21.15.31:6826/573482428,v1:172.21.15.31:6827/573482428] exists,up 2545300a-428a-41ed-b19a-3a0cb9e5e755', 'osd.4 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6801/3424463091,v1:172.21.15.149:6803/3424463091] [v2:172.21.15.149:6805/3424463091,v1:172.21.15.149:6807/3424463091] exists,up 886aeed5-4495-4064-8579-050d38a4ad1b', 'osd.5 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6816/3709461555,v1:172.21.15.149:6818/3709461555] [v2:172.21.15.149:6820/3709461555,v1:172.21.15.149:6822/3709461555] exists,up 4d15cef0-9ee9-4069-a6c4-a650e873ee03', 'osd.6 up in weight 1 up_from 13 up_thru 14 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6800/405561966,v1:172.21.15.149:6802/405561966] [v2:172.21.15.149:6804/405561966,v1:172.21.15.149:6806/405561966] exists,up 910ac664-9573-4851-9e2b-89ac6015fdc1', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.149:6817/570236217,v1:172.21.15.149:6819/570236217] [v2:172.21.15.149:6821/570236217,v1:172.21.15.149:6823/570236217] exists,up 14cc2d94-48e4-47c9-896a-9be9281ef0c5'] 2024-05-08T08:25:02.502 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "name": "mon.a", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster": "ceph", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "admin_socket_mode": "", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_client_required": "cephx, none", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_cluster_required": "cephx", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_debug": "true", 2024-05-08T08:25:02.607 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_required": "cephx", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_service_ticket_ttl": "240.000000", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "auth_supported": "cephx", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio": "true", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_max_queue_depth": "1024", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_poll_ms": "250", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_aio_reap_max": "16", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_async_discard": "false", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_block_size": "4096", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio": "false", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-05-08T08:25:02.608 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_debug_inflight_ios": "false", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_enable_discard": "false", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry": "3", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_flock_retry_interval": "0.100000", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash": "0", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_inject_crash_flush_delay": "2", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring": "false", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_hipri": "false", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_ioring_sqthread_poll": "false", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_buffer_alignment": "4096", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bdev_type": "", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_alloc_size": "1048576", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_allocator": "hybrid", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_buffered_io": "true", 2024-05-08T08:25:02.609 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_for_zeros": "false", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_often": "false", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_compact_log_sync": "false", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_compact_min_size": "16777216", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_log_replay_check_allocations": "true", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_log_runway": "4194304", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_max_prefetch": "1048576", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_flush_size": "524288", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_min_log_runway": "1048576", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery": "false", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_shared_alloc_size": "65536", 2024-05-08T08:25:02.610 INFO:teuthology.orchestra.run.smithi031.stdout: "bluefs_sync_write": "false", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocation_from_file": "true", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_allocator": "hybrid", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_blobid_prealloc": "10240", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_create": "true", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_create": "false", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_path": "", 2024-05-08T08:25:02.611 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_db_size": "0", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_path": "", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_preallocate_file": "false", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_size": "107374182400", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_create": "false", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_path": "", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_block_wal_size": "100663296", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs": "true", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_env_mirror": "false", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-05-08T08:25:02.612 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune": "true", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size": "0", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cache_type": "2q", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_clone_cow": "true", 2024-05-08T08:25:02.613 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_algorithm": "snappy", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size": "0", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size": "0", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_mode": "none", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_csum_type": "crc32c", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_enforce_settings": "default", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_freelist": "false", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_fsck_abort": "false", 2024-05-08T08:25:02.614 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_inject_read_err": "false", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_legacy_omap": "false", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefill": "0.000000", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_small_allocations": "0", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_read": "true", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_default_buffered_write": "false", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops": "0", 2024-05-08T08:25:02.615 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_elastic_shared_blobs": "true", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fail_eio": "false", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs": "true", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount": "false", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-05-08T08:25:02.616 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount": "false", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_ignore_data_csum": "false", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_kvbackend": "rocksdb", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_log_op_age": "5.000000", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_alloc_size": "0", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size": "0", 2024-05-08T08:25:02.617 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_defer_interval": "3.000000", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_max_deferred_txc": "32", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size": "0", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_nid_prealloc": "1024", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size": "0", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_qfsck_on_mount": "true", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_retry_disk_reads": "3", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cf": "true", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-05-08T08:25:02.618 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_rocksdb_options_annex": "", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_coremask": "0x1", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_io_sleep": "5", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_max_io_completion": "0", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_spdk_mem": "512", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_sync_submit_transaction": "false", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_bytes": "67108864", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io": "0", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_tracing": "false", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-05-08T08:25:02.619 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved": "0", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "bluestore_zero_block_detection": "false", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_action_update_interval": "2", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-05-08T08:25:02.620 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_mount_timeout": "10", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_signatures": "false", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_cluster_require_version": "2", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_signatures": "false", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_require_version": "2", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_signatures": "false", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_service_require_version": "2", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "cephx_sign_messages": "true", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "chdir": "", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_acl_type": "", 2024-05-08T08:25:02.621 INFO:teuthology.orchestra.run.smithi031.stdout: "client_asio_thread_count": "2", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_mid": "0.750000", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_cache_size": "16384", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_caps_release_delay": "5", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_check_pool_perm": "true", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_collect_and_send_global_metrics": "false", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_force_sync_read": "false", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_getattr_caps": "false", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_debug_inject_tick_delay": "0", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_die_on_failed_remount": "false", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_dirsize_rbytes": "true", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_force_lazyio": "false", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_fs": "", 2024-05-08T08:25:02.622 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_fixed_oldest_tid": "false", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_inject_release_failure": "false", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_inline_size": "4096", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_max_retries_on_remount_failure": "5", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mds_namespace": "", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_metadata": "", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_gid": "-1", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_timeout": "300", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mount_uid": "-1", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_mountpoint": "/", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_notify_timeout": "10", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc": "true", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty": "104857600", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_dirty_age": "5.000000", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_max_objects": "1000", 2024-05-08T08:25:02.623 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_size": "209715200", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_oc_target_dirty": "8388608", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_permissions": "true", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota": "true", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_quota_df": "true", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_bytes": "0", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_max_periods": "4", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_readahead_min": "131072", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_reconnect_stale": "false", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_shutdown_timeout": "30", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_snapdir": ".snap", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_tick_interval": "1", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_trace": "", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_try_dentry_invalidate": "false", 2024-05-08T08:25:02.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_faked_inos": "false", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_use_random_mds": "false", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog": "false", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_graylog_port": "12201", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_monitors": "default=true", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog": "false", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "clog_to_syslog_level": "info", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "-", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network": "", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_network_interface": "", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_isal": "false", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_level": "5", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zlib_winsize": "-15", 2024-05-08T08:25:02.625 INFO:teuthology.orchestra.run.smithi031.stdout: "compressor_zstd_level": "1", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_op_num_threads": "6", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_alien_thread_cpu_cores": "", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_obc_lru_size": "512", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crimson_seastar_cpu_cores": "", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location": "", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook": "", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_location_hook_timeout": "10", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "daemonize": "false", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "daos_pool": "tank", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-05-08T08:25:02.626 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "dbstore_db_name_prefix": "dbstore", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_alienstore": "0/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_allow_any_pool_priority": "false", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok": "1/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asok_assert_abort": "false", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_asserts_on_shutdown": "true", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_auth": "20/20", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bdev": "1/3", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluefs": "1/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_bluestore": "1/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_buffer": "0/1", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ceph_exporter": "1/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephfs_mirror": "0/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cephsqlite": "0/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_client": "0/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_compressor": "1/5", 2024-05-08T08:25:02.627 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_context": "0/1", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crimson_interrupt": "0/5", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crush": "1/1", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_crypto": "1/5", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_cyanstore": "0/5", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_deliberately_leak_memory": "false", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_disable_randomized_ping": "false", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_dpdk": "1/5", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_eventtrace": "1/5", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filer": "0/1", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_filestore": "1/3", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_finisher": "1/1", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_fuse": "1/5", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeat_testing_span": "0", 2024-05-08T08:25:02.628 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_heartbeatmap": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_immutable_obj_cache": "0/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_javaclient": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journal": "1/3", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_journaler": "0/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_kstore": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_lockdep": "0/1", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mclock": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_balancer": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_locker": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_log_expire": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_migrator": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mds_quiesce": "3/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_memstore": "1/5", 2024-05-08T08:25:02.629 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgr": "2/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mgrc": "1/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_mon": "20/20", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_monc": "0/10", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_ms": "1/1", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_none": "0/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objclass": "0/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objectcacher": "0/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_objecter": "0/1", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_optracker": "0/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_osd": "1/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_paxos": "20/20", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_perfcounter": "1/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_prioritycache": "1/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rados": "0/5", 2024-05-08T08:25:02.630 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd": "0/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_mirror": "0/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_pwl": "0/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rbd_replay": "0/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_refs": "0/0", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_reserver": "1/1", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw": "1/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_access": "1/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_datacache": "1/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_dbstore": "1/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_flight": "1/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_lifecycle": "1/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rgw_sync": "1/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_rocksdb": "4/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore": "0/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_backref": "0/5", 2024-05-08T08:25:02.631 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cache": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_cleaner": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_device": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_epm": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_journal": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_lba": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_odata": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_omap": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_onode": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_t": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_seastore_tm": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_striper": "0/1", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_test": "0/5", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_throttle": "1/1", 2024-05-08T08:25:02.632 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_timer": "0/1", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_tp": "0/5", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "debug_trace": "1/5", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "device_failure_prediction_mode": "none", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_graylog": "false", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_journald": "false", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_stderr": "true", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "err_to_syslog": "false", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event_tracing": "false", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_addr": "0.0.0.0", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_http_port": "9926", 2024-05-08T08:25:02.633 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_prio_limit": "5", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_sort_metrics": "true", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "exporter_stats_period": "5", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "fake_statfs_for_testing": "0", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "fatal_signal_handlers": "true", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_purge_ops": "10", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filer_max_truncate_ops": "128", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_apply_finisher_threads": "1", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_blackhole": "false", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_clone_range": "true", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_btrfs_snap": "true", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_caller_concurrency": "10", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_collect_device_partition_information": "true", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_commit_timeout": "600.000000", 2024-05-08T08:25:02.634 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_inject_read_err": "false", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_omap_check": "false", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_random_read_err": "0.000000", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_debug_verify_split": "false", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_dump_file": "", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fadvise": "true", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fail_eio": "true", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_shards": "16", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fd_cache_size": "128", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap": "false", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fiemap_threshold": "4096", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_index_retry_probability": "0.000000", 2024-05-08T08:25:02.635 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_inject_stall": "0", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_parallel": "false", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_trailing": "false", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_journal_writeahead": "false", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_kill_at": "0", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size": "0", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs": "0", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_other": "2", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-05-08T08:25:02.636 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_sync_interval": "5.000000", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size": "0", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_merge_threshold": "-10", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_min_sync_interval": "0.010000", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_odsync_write": "false", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend": "rocksdb", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_backend_path": "", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_omap_header_cache_size": "1024", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_ondisk_finisher_threads": "1", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_thread_timeout": "60", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_op_threads": "2", 2024-05-08T08:25:02.637 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_punch_hole": "false", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_bytes": "104857600", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_queue_max_ops": "50", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_seek_data_hole": "false", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc": "false", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-05-08T08:25:02.638 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_splice": "false", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_multiple": "2", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_split_rand_factor": "20", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_update_to": "1000", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_enable": "true", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-05-08T08:25:02.639 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_xfs_extsize": "false", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "filestore_zfs_snap": "false", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fio_dir": "/tmp/fio", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_allow_other": "true", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_atomic_o_trunc": "true", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_big_writes": "true", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_debug": "false", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_default_permissions": "false", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_disable_pagecache": "false", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_max_write": "0", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_multithreaded": "true", 2024-05-08T08:25:02.640 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_require_active_mds": "true", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_set_user_groups": "true", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_move": "true", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_read": "true", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_splice_write": "true", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_syncfs_on_mksnap": "true", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "fuse_use_invalidate_cb": "true", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "gss_target_name": "ceph", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_file": "", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_inject_failure": "0", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_interval": "5", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "host": "smithi031", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-05-08T08:25:02.641 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_max_size": "1073741824", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "immutable_object_cache_watermark": "0.900000", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "inject_early_sigterm": "false", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_agent_port": "6799", 2024-05-08T08:25:02.642 INFO:teuthology.orchestra.run.smithi031.stdout: "jaeger_tracing_enable": "false", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_aio": "true", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_align_min_size": "65536", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_align": "true", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_block_size": "4096", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_dio": "true", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_discard": "false", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_force_aio": "false", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_ignore_corruption": "false", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_bytes": "10485760", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_max_write_entries": "100", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_replay_from": "0", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_multiple": "0.000000", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-05-08T08:25:02.643 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_throttle_max_multiple": "0.000000", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_write_header_frequency": "0", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "journal_zero_on_create": "false", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prefetch_periods": "10", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_prezero_periods": "5", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "journaler_write_head_interval": "15", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "key": "", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "keyfile": "", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_backend": "rocksdb", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_default_stripe_size": "65536", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount": "false", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_fsck_on_mount_deep": "true", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_bytes": "67108864", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_max_ops": "512", 2024-05-08T08:25:02.644 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_nid_prealloc": "1024", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_onode_map_size": "1024", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_submit_transaction": "false", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "kstore_sync_transaction": "false", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "librados_thread_count": "2", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep": "false", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "lockdep_force_backtrace": "false", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_coarse_timestamps": "true", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_flush_on_exit": "false", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_host": "127.0.0.1", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_graylog_port": "12201", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_new": "1000", 2024-05-08T08:25:02.645 INFO:teuthology.orchestra.run.smithi031.stdout: "log_max_recent": "10000", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stderr_prefix": "", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_stop_at_utilization": "0.970000", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_file": "true", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_graylog": "false", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_journald": "false", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_stderr": "false", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "log_to_syslog": "false", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "max_rotating_auth_attempts": "10", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_action_on_write_error": "1", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_alternate_name_max": "8192", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_asio_thread_count": "2", 2024-05-08T08:25:02.646 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_export_pin": "true", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_dirs": "true", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_interval": "5", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_fragment_size_max": "100000", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_idle_threshold": "0.000000", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_interval": "10", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max": "-1", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_max_until": "-1", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_merge_size": "50", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_midchunk": "0.300000", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_rebalance": "0.100000", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_min_start": "0.200000", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_minchunk": "0.001000", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_mode": "0", 2024-05-08T08:25:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_max": "1.200000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_need_min": "0.800000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_overload_epochs": "2", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_sample_interval": "3.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_bits": "3", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_rd": "25000.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_size": "10000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_split_wr": "10000.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_target_decay": "10.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_grace": "15.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_interval": "4.000000", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_beacon_mon_down_grace": "60", 2024-05-08T08:25:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_memory_limit": "4294967296", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_mid": "0.700000", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_delay": "0", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_sleep": "200", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_splitauth": "true", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_quiesce_threshold": "524288", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_release_free_interval": "10", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_reservation": "0.050000", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_interval": "1", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cache_trim_threshold": "262144", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_delegate_inos_pct": "50", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_prealloc_inos": "1000", 2024-05-08T08:25:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_connect_bootstrapping": "false", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_damage_table_max_entries": "10000", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_auth_pins": "false", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_frag": "false", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_scatterstat": "false", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_debug_subtrees": "false", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_decay_halflife": "5.000000", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_default_dir_hash": "2", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_defer_session_stale": "true", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_deny_all_reconnect": "false", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_keys_per_op": "16384", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_commit_size": "10", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_max_entries": "0", 2024-05-08T08:25:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dir_prefetch": "true", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dirstat_min_interval": "1.000000", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_after_rejoin": "false", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_on_map": "false", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_file": "0", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_early_reply": "true", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enable_op_tracker": "true", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_enforce_unique_name": "true", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed": "true", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random": "true", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-05-08T08:25:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_forward_all_requests_to_auth": "false", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_cache_threshold": "1.500000", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_health_summarize_threshold": "10", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_grace": "15.000000", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_heartbeat_reset_grace": "1000", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_health_dummy": "false", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_migrator_session_race": "false", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-05-08T08:25:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_join_fs": "", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_journal_format": "1", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_create_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_export_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_import_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_expire_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_journal_replay_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_link_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_mdstable_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_openc_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_rename_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_shutdown_at": "0", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_event_large_threshold": "524288", 2024-05-08T08:25:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_events_per_segment": "1024", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_major_segment_event_ratio": "12", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_events": "-1", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_max_segments": "128", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_pause": "false", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_segment_size": "0", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_corrupt_events": "false", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_skip_unbounded_events": "false", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_threshold": "128", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_log_warn_factor": "2.000000", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_caps_per_client": "1048576", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_flushes": "100000", 2024-05-08T08:25:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_completed_requests": "100000", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_export_size": "20971520", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_file_recover": "32", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_files": "64", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops": "8192", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_max_snaps_per_dir": "100", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_metrics_update_interval": "2", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_per_client": "100", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_min_caps_working_set": "10000", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_numa_node": "-1", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_complaint_time": "30.000000", 2024-05-08T08:25:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_duration": "600", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_size": "20", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_size": "20", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_history_slow_op_threshold": "10", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_op_log_threshold": "5", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_grace": "15", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_ping_interval": "5", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_caps": "30000", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_max_decay_threshold": "131072", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_recall_warning_threshold": "262144", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_reconnect_timeout": "45.000000", 2024-05-08T08:25:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_interval": "1.000000", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_gid": "0", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_root_ino_uid": "0", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_evict": "true", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_blocklist_on_timeout": "true", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_session_metadata_threshold": "16777216", 2024-05-08T08:25:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_shutdown_check": "0", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_skip_ino": "0", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_sleep_rank_change": "0.000000", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_max_uid": "4294967294", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_min_uid": "0", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_snap_rstat": "false", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_standby_replay_damaged": "false", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_symlink_recovery": "true", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_task_status_update_interval": "2.000000", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_exports": "0", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_thrash_fragments": "0", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_tick_interval": "5.000000", 2024-05-08T08:25:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_valgrind_exit": "false", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_backtrace": "1", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_verify_scatter": "false", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_ino_prealloc": "false", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mds_wipe_sessions": "false", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mempool_debug": "false", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_debug_omit_block_device_write": "false", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_device_bytes": "1073741824", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_set": "false", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "memstore_page_size": "65536", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_bytes": "134217728", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_messages": "512", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_connect_retry_interval": "1.000000", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-05-08T08:25:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_disabled_modules": "", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_max_pg_num_change": "128", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_bytes": "134217728", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mds_messages": "128", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_bytes": "134217728", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_mon_messages": "128", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_bytes": "536870912", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_osd_messages": "8192", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_pool": "true", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_service_beacon_grace": "60.000000", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_standby_modules": "true", 2024-05-08T08:25:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_period": "5", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_stats_threshold": "5", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_tick_period": "2", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_accept_timeout_factor": "2.000000", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_delete": "true", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_allow_pool_size_one": "true", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_auth_validate_all_caps": "true", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_bytes": "104857600", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_directed_command_retry": "5", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval": "3.000000", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-05-08T08:25:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_hunt_parallel": "3", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_log_interval": "1.000000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_interval": "10.000000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_ping_timeout": "30.000000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_client_target_rank": "-1", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_allowed": "1.000000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_file_level": "debug", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_file": "true", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog": "false", 2024-05-08T08:25:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_journald": "false", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_stderr": "false", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_bootstrap": "false", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_start": "false", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_compact_on_trim": "true", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_persist_interval": "10", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_con_tracker_score_halflife": "43200", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_config_key_max_entry_size": "65536", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_cpu_threads": "4", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_crush_min_required_version": "hammer", 2024-05-08T08:25:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_daemon_bytes": "419430400", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_crit": "5", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_avail_warn": "5", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_data_size_warn": "16106127360", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_block_osdmap_trim": "false", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_json": "false", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_dump_transactions": "false", 2024-05-08T08:25:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_extra_checks": "false", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_reef": "false", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_no_require_squid": "false", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_delta_reset_interval": "10.000000", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_dns_srv_name": "ceph-mon", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_mkfs_grace": "120", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_down_uptime_grace": "60", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_default_strategy": "1", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_election_timeout": "5.000000", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_divisor": "2", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_elector_ping_timeout": "2.000000", 2024-05-08T08:25:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_enable_op_tracker": "true", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_fake_pool_delete": "false", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_force_quorum_join": "false", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_globalid_prealloc": "10000", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_detail_to_clog": "false", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_log_update_period": "5", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_max_detail": "50", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog": "true", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_interval": "600", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host": "172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790],172.21.15.149", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_host_override": "", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_initial_members": "", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-05-08T08:25:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_keyvaluedb": "rocksdb", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease": "5.000000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_full_interval": "50", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max": "10000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_log_max_summary": "50", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_entries_per_event": "4096", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_log_epochs": "500", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mdsmap_epochs": "500", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_mgrmap_epochs": "500", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_osd": "10000", 2024-05-08T08:25:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pg_per_osd": "10000", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_pool_pg_num": "65536", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_force_trim_to": "0", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mds_skip_sanity": "false", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_autotune": "true", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_memory_target": "2147483648", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_beacon_grace": "30", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_digest_period": "5", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_inactive_grace": "60", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_mkfs_grace": "240", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-05-08T08:25:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_min_osdmap_epochs": "500", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_complaint_time": "30", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_duration": "600", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_size": "20", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_size": "20", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_history_slow_op_threshold": "10", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_op_log_threshold": "5", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_in": "false", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_auto_mark_new_in": "true", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-05-08T08:25:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size": "500", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_cache_size_min": "134217728", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_crush_smoke_test": "true", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_destroyed_out_interval": "600", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_interval": "600", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_force_trim_to": "0", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_full_ratio": "0.950000", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_halflife": "3600", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_max_interval": "300", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_laggy_weight": "0.300000", 2024-05-08T08:25:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_creating_pgs": "1024", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_max_initial_pgs": "1024", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_down_reporters": "2", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp": "true", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_report_timeout": "900", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_num_repaired": "10", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osd_warn_op_age": "32.000000", 2024-05-08T08:25:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_interval": "10", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_min": "10000", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_stuck_threshold": "60", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_objects": "10000", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_per_osd": "0", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_crit_threshold": "0", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_pool_quota_warn_threshold": "0", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_probe_timeout": "2.000000", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_change": "0.050000", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_max_osds": "4", 2024-05-08T08:25:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_interval": "86400", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_max_keys": "100", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_scrub_timeout": "300", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_session_timeout": "300", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_smart_report_timeout": "5", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stat_smooth_intervals": "6", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_min_size": "2", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_pool_size": "4", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-05-08T08:25:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_subscribe_interval": "86400.000000", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_debug": "false", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_keys": "2000", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_max_payload_size": "1048576", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_provider_kill_at": "0", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_requester_kill_at": "0", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_sync_timeout": "60.000000", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_target_pg_per_osd": "100", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_tick_interval": "5", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_interval": "300.000000", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_use_min_delay_socket": "false", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_older_version_delay": "604800", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-05-08T08:25:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_filestore_osds": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_misplaced": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_older_version": "true", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_app": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-05-08T08:25:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_on_too_few_osds": "false", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "monmap": "", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "motr_tracing_enabled": "false", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_op_threads": "3", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_buffer_size": "131072", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_cm": "false", 2024-05-08T08:25:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_device_name": "", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_dscp": "96", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_gid_idx": "0", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_local_gid": "", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_polling_us": "1000", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_port_num": "1", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_roce_ver": "1", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_send_buffers": "1024", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_sl": "3", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_support_srq": "true", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_rdma_type": "ib", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_async_reap_threshold": "5", 2024-05-08T08:25:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_before_connect": "false", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv4": "true", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_ipv6": "false", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr1": "true", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_msgr2": "true", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_max": "7568", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_port_min": "6800", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_prefer_ipv4": "false", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_count": "3", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_bind_retry_delay": "5", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_client": "false", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mds": "false", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mgr": "false", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_mon": "false", 2024-05-08T08:25:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_blackhole_osd": "false", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_client_mode": "crc secure", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_mode": "crc secure", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_cluster_type": "", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_compress_secure": "false", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_idle_timeout": "900", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_connection_ready_timeout": "10", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_data": "true", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_crc_header": "true", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bad_msg": "false", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_bug": "true", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_old_message": "true", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_skipped_message": "false", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_die_on_unhandled_msg": "false", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-05-08T08:25:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_coremask": "0xF", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_devs_allowlist": "", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_enable_tso": "true", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hugepages": "", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_flow_control": "true", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_lro": "true", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_memory_channel": "4", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_pmd": "", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_port_id": "0", 2024-05-08T08:25:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_corrupt_message_level": "1", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_dump_on_send": "false", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_initial_backoff": "0.200000", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_max": "1.000000", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_probability": "0.000000", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_delay_type": "", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_internal_delays": "0.000000", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_network_congestion": "0", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_inject_socket_failures": "5000", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_learn_addr_from_peer": "true", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_accept_failures": "4", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_max_backoff": "15.000000", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_client_mode": "secure crc", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_cluster_mode": "secure crc", 2024-05-08T08:25:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_mon_service_mode": "secure crc", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_min_size": "1024", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compress_mode": "none", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_osd_compression_algorithm": "snappy", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_pq_min_cost": "65536", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_public_type": "", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_service_mode": "crc secure", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_listen_backlog": "512", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_nodelay": "true", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_tcp_rcvbuf": "0", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "ms_type": "async+posix", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "no_config_file": "false", 2024-05-08T08:25:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_completion_locks_per_session": "32", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_debug_inject_relock_delay": "false", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_op_bytes": "104857600", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inflight_ops": "1024", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_inject_no_watch_ping": "false", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_tick_interval": "5.000000", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objecter_timeout": "10.000000", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_blackhole": "false", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "openssl_engine_opts": "", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_delay_time": "5.000000", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_hist_halflife": "1000", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_low_ops": "2", 2024-05-08T08:25:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_max_ops": "4", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_quantize_effort": "0.100000", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_agent_slop": "0.020000", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_allow_recovery_below_min_size": "true", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_asio_thread_count": "2", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_async_recovery_min_cost": "100", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_auto_mark_unfound_lost": "false", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_retry_interval": "30.000000", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_max": "512", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backfill_scan_min": "64", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_degraded": "false", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_peering": "false", 2024-05-08T08:25:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_backoff_on_unfound": "true", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_beacon_report_interval": "300", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_duration": "30", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_max_block_size": "67108864", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_bench_small_size_max_iops": "100", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blkin_trace_all": "false", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_blocked_scrub_grace_period": "120", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_for_log_corruption": "false", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T08:25:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_class_update_on_start": "true", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_cap": "256", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_message_size_cap": "524288000", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_op_priority": "63", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_client_watch_timeout": "30", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_command_max_records": "256", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_compact_on_start": "false", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_chooseleaf_type": "0", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_initial_weight": "-1.000000", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_on_start": "true", 2024-05-08T08:25:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_crush_update_weight_set": "true", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_duration": "0", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_feed_pullee": "-1", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_misdirected_ops": "false", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_acting_change": "false", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_no_purge_strays": "false", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_op_order": "false", 2024-05-08T08:25:02.687 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pg_log_writeout": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_pretend_recovery_active": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_shutdown": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_trim_objects": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_cached_snaps": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_missing_on_start": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_snaps": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_keys": "1024", 2024-05-08T08:25:02.688 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_stride": "524288", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_data_pool_replay_window": "5", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_default_notify_timeout": "30", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep": "0.000000", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_discard_disconnected_ops": "true", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_enable_op_tracker": "true", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_extblkdev_plugins": "vdo", 2024-05-08T08:25:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_info": "true", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown": "true", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_fast_shutdown_timeout": "15", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_function_tracing": "false", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_grace": "20", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_interval": "6", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_peers": "10", 2024-05-08T08:25:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_min_size": "2000", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_stale": "600", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_max_size": "100000", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_min_size": "1000", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_flush_on_shutdown": "true", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_journal_size": "5120", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_kill_backfill_at": "0", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_loop_before_reset_tphandle": "64", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_cache_size": "50", 2024-05-08T08:25:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_dedup": "true", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max": "40", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_message_max_bytes": "10485760", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_map_share_max_epochs": "40", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_name_len": "100", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_attr_size": "0", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_backfills": "1", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_count": "5", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_markdown_period": "600", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_name_len": "2048", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_namespace_len": "256", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_object_size": "134217728", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_omap_entries_per_request": "1024", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_blocked_by": "16", 2024-05-08T08:25:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_log_entries": "10000", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_pgls": "1024", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_cost": "8388608", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_push_objects": "10", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_scrubs": "3", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_trimming_pgs": "2", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_op_reply_len": "64", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_max_write_size": "90", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-05-08T08:25:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_override_recovery_settings": "false", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_profile": "balanced", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-05-08T08:25:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mclock_skip_benchmark": "false", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_base": "805306368", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_min": "134217728", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target": "4294967296", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_autotune": "false", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_pg_log_entries": "250", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_min_recovery_priority": "0", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_interval": "30", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_interval": "5", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_report_max_in_flight": "2", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_cache_shards": "32", 2024-05-08T08:25:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_num_op_tracker_shard": "32", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_auto_affinity": "true", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_node": "-1", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_numa_prefer_iface": "true", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objecter_finishers": "1", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore": "bluestore", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_fuse": "false", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_objectstore_tracing": "false", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_complaint_time": "30.000000", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_duration": "600", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_size": "20", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_size": "20", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_log_threshold": "5", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards": "0", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_hdd": "5", 2024-05-08T08:25:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_shards_ssd": "8", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard": "0", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_pq_min_cost": "65536", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue": "mclock_scheduler", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_queue_cut_off": "high", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_suicide_timeout": "150", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_op_thread_timeout": "15", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_open_classes_on_start": "true", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_os_flags": "0", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_peering_op_priority": "255", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_cost": "1048576", 2024-05-08T08:25:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_delete_priority": "5", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_dups_tracked": "3000", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_max": "10000", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_log_trim_min": "100", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_object_context_cache_count": "64", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-05-08T08:25:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crimson": "false", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_crush_rule": "-1", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_ec_fast_read": "false", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_bulk": "false", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nodelete": "false", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_flags": "0", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_min_size": "0", 2024-05-08T08:25:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pg_num": "32", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_pgp_num": "0", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_read_ratio": "70", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_size": "2", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_default_type": "replicated", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_pool_use_gmt_hitset": "true", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_push_per_object_cost": "1000", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_read_ec_check_for_errors": "false", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap": "true", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recover_clone_overlap_limit": "10", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_cost": "20971520", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_delay_start": "0.000000", 2024-05-08T08:25:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active": "0", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_hdd": "3", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_active_ssd": "10", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_chunk": "8388608", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_max_single_start": "1", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_priority": "3", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_op_warn_multiple": "16", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_priority": "5", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_retry_interval": "30.000000", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep": "0.000000", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-05-08T08:25:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_repair_during_recovery": "false", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_requested_scrub_priority": "120", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_rollback_to_cluster_snap": "", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair": "false", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_hour": "0", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_begin_week_day": "0", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_max": "25", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_chunk_min": "5", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_cost": "52428800", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_during_recovery": "false", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_hour": "0", 2024-05-08T08:25:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_end_week_day": "0", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_event_cost": "4096", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_invalid_stats": "true", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_load_threshold": "0.500000", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_interval": "604800.000000", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_max_preemptions": "5", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_min_interval": "86400.000000", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_priority": "5", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_scrub_sleep": "0.000000", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_shutdown_pgref_assert": "false", 2024-05-08T08:25:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_skip_data_digest": "false", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_smart_report_timeout": "5", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_cost": "1048576", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_priority": "5", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep": "0.000000", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_stats_update_period_scrubbing": "15", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_target_transaction_size": "30", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-05-08T08:25:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_default_cache_mode": "writeback", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_tracing": "false", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_use_stale_snap": "false", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "osdc_blkin_trace_all": "false", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_kill_at": "0", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_max_join_drift": "10", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min": "500", 2024-05-08T08:25:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_min_wait": "0.050000", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_propose_interval": "1.000000", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max": "500", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_max_multiplier": "20", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_service_trim_min": "250", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_stash_full_interval": "25", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_max": "500", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "paxos_trim_min": "250", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "perf": "true", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "-", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrv": "", 2024-05-08T08:25:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "public_bind_addr": "-", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network": "", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "public_network_interface": "", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_enabled": "false", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "qat_compressor_session_max_number": "256", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_mon_op_timeout": "0", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_osd_op_timeout": "0", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rados_tracing": "false", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_atime_update_interval": "60", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_parent_reads": "false", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_balance_snap_reads": "false", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blkin_trace_all": "false", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_expire_seconds": "0", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_blocklist_on_break_lock": "true", 2024-05-08T08:25:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache": "true", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_block_writes_upfront": "false", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty": "25165824", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_max_dirty_object": "0", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_policy": "writearound", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_size": "33554432", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_target_dirty": "16777216", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_clone_copy_on_read": "false", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_compression_hint": "none", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_concurrent_management_ops": "10", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_clone_format": "auto", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_data_pool": "", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_features": "61", 2024-05-08T08:25:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_format": "2", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_map_options": "", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_order": "22", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_pool": "rbd", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_count": "0", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_default_stripe_unit": "0", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_disable_zero_copy_writes": "true", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_granularity_bytes": "65536", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_enable_alloc_hint": "true", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler": "simple", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_commit_age": "5.000000", 2024-05-08T08:25:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_flush_interval": "0", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_order": "24", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_pool": "", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_journal_splay_width": "4", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_parent_reads": "false", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_localize_snap_reads": "false", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-05-08T08:25:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_type": "simple", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_autotune": "true", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_base": "805306368", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-05-08T08:25:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_memory_target": "4294967296", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_delete_delay": "0", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_replay_delay": "0", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove": "false", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_mtime_update_interval": "60", 2024-05-08T08:25:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_non_blocking_aio": "true", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_thread_timeout": "60", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_op_threads": "1", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_parent_cache_enabled": "false", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_mode": "disabled", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_path": "/tmp", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_persistent_cache_size": "1073741824", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_plugins": "", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst": "0", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_bps_limit": "0", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_exclude_ops": "0", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst": "0", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_iops_limit": "0", 2024-05-08T08:25:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_bps_limit": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_read_iops_limit": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_schedule_tick_min": "50", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_bps_limit": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_qos_write_iops_limit": "0", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_quiesce_notification_attempts": "10", 2024-05-08T08:25:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_read_from_replica_policy": "default", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_max_bytes": "524288", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_readahead_trigger_requests": "10", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_request_timed_out_seconds": "30", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_skip_partial_discard": "true", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_tracing": "false", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_names": "true", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd_validate_pool": "true", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_base_url": "", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "restapi_log_level": "", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_account_default_quota_max_size": "-1", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_acl_grants_max_num": "100", 2024-05-08T08:25:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_admin_entry": "admin", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_backend_store": "rados", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_barbican_url": "", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_beast_enable_async": "true", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache": "false", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_max_aio": "128", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_quota_ttl": "600", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_enabled": "true", 2024-05-08T08:25:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_expiry_interval": "900", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cache_lru_size": "25000", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_config_store": "rados", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_content_length_compat": "false", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress": "true", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cors_rules_max_num": "100", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_cross_domain_policy": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_default_encryption_key": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_addr": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_ca_path": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_cert": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_client_key": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_password": "", 2024-05-08T08:25:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_kmip_username": "", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_require_ssl": "true", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-05-08T08:25:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_suppress_logs": "true", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_addr": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_auth": "token", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_namespace": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_prefix": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_token_file": "", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_buffersize": "524288", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_limit": "1024", 2024-05-08T08:25:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_low_speed_time": "300", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_tcp_keepalive": "0", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_fadvise": "4", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_host": "127.0.0.1", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_d4n_port": "6379", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_changes_size": "1000", 2024-05-08T08:25:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_num_shards": "128", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_log_window": "30", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_notify_interval_msec": "0", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_poll_interval": "20", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_data_sync_spawn_window": "20", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_data_log_backing": "fifo", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_region_info_oid": "default.region", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_defer_to_bucket_acls": "", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-05-08T08:25:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_res": "500.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_name": "", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dns_s3website_name": "", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_dynamic_resharding": "true", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-05-08T08:25:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_gc_threads": "true", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_lc_threads": "true", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_ops_log": "false", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_quota_threads": "true", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_static_website": "false", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enable_usage_log": "false", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_enforce_swift_acls": "true", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_exit_timeout_secs": "120", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_expose_bucket": "false", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_extended_http_attrs": "", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_filter": "none", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_frontends": "beast port=7480", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_concurrent_io": "10", 2024-05-08T08:25:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred": "50", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_objs": "32", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_queue_size": "134213632", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_max_trim_chunk": "16", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_obj_min_wait": "7200", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_max_time": "3600", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_gc_processor_period": "3600", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_get_obj_window_size": "16777216", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_healthcheck_disabling_path": "", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ignore_get_invalid_range": "false", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_init_timeout": "300", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-05-08T08:25:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_connection_idle": "300", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_message_timeout": "5000", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_kafka_sleep_timeout": "10", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_domain": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_password_path": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_project": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_tenant": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_token_path": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_admin_user": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_api_version": "2", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_domain": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_password": "", 2024-05-08T08:25:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_project": "", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_tenant": "", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_barbican_user": "", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_implicit_tenants": "false", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_service_token_enabled": "false", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_token_cache_size": "10000", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_url": "", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_keystone_verify_ssl": "true", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_debug_interval": "-1", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_lock_max_time": "90", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_objs": "32", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_rules": "1000", 2024-05-08T08:25:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_worker": "3", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_max_wp_worker": "3", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lc_thread_delay": "0", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_dnattr": "uid", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_searchfilter": "", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ldap_uri": "", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_http_headers": "", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_nonexistent_bucket": "false", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-05-08T08:25:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_log_object_name_utc": "false", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_name_len": "0", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attr_size": "0", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_attrs_num_in_req": "0", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_chunk_size": "4194304", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_concurrent_requests": "1024", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_dynamic_shards": "1999", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_listing_results": "1000", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_notify_retries": "10", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_objs_per_shard": "100000", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_param_size": "1048576", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_put_size": "5368709120", 2024-05-08T08:25:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_max_slo_entries": "1000", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_log_max_shards": "64", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_md_notify_interval_msec": "200", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_poll_interval": "20", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_meta_sync_spawn_window": "20", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_mp_lock_max_time": "600", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_min_part_size": "5242880", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_fhcache_size": "2017", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_lru_lanes": "5", 2024-05-08T08:25:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_max_gc": "300", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_gc_threads": "false", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_lc_threads": "false", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_quota_threads": "false", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_run_sync_thread": "false", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_async_rados_threads": "32", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_num_control_oids": "8", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_numa_node": "-1", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_stripe_size": "4194304", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_chunk_size": "100", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_gc_interval": "600", 2024-05-08T08:25:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_objexp_hints_num_shards": "127", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_thread_timeout": "600", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_op_tracing": "false", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_token": "", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_url": "", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_opa_verify_ssl": "true", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_rados": "false", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_ops_log_socket_path": "", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval": "2.000000", 2024-05-08T08:25:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_push_interval_max": "30.000000", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_period_root_pool": ".rgw.root", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lanes": "3", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_max_buckets": "100", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_cache_partitions": "3", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_continue": "true", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_print_prohibited_content_length": "false", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-05-08T08:25:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rados_tracing": "false", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm": "", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_id": "", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region": "", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_region_root_pool": ".rgw.root", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_region_enforcement": "false", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_request_uri": "", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_batch_size": "64", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_max_aio": "128", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_num_logs": "16", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_reshard_thread_interval": "600", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_resolve_cname": "false", 2024-05-08T08:25:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_rest_getusage_op_compat": "false", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_run_sync_thread": "true", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_keystone": "false", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_ldap": "false", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_rados": "true", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_auth_use_sts": "false", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_s3_success_create_obj_status": "0", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_scheduler_type": "throttler", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_script_uri": "", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_service_provider_name": "", 2024-05-08T08:25:02.734 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_id": "", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_client_secret": "", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_entry": "sts", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_key": "sts", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_max_session_duration": "43200", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_min_session_duration": "900", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sts_token_introspection_url": "", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_account_in_url": "false", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_entry": "auth", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_auth_url": "", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_custom_header": "", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_enforce_content_length": "false", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_need_stats": "true", 2024-05-08T08:25:02.735 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_tenant_name": "", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_token_expiration": "86400", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url": "", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_url_prefix": "swift", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_swift_versioning_enabled": "false", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_lease_period": "120", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_interval": "1200", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_obj_etag_verify": "false", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_history_size": "4096", 2024-05-08T08:25:02.736 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_thread_pool_size": "512", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_max_retries": "0", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_topic_require_publish_policy": "false", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_comment": "", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_createby": "", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_encoding": "", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_flag": "false", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_max_size": "5368709120", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_origin": "", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_sha_unit": "524288", 2024-05-08T08:25:02.737 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_torrent_tracker": "", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_trust_forwarded_https": "false", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_log_tick_interval": "30", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_shards": "32", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_usage_max_user_shards": "1", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_use_opa_authz": "false", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache": "false", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_counters_cache_size": "10000", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_default_quota_max_size": "-1", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_max_buckets": "1000", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_policies_max_num": "100", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-05-08T08:25:02.738 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_interval": "86400", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_user_unique_email": "true", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_verify_ssl": "true", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_website_routing_rules_max_num": "50", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone": "", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_id": "", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup": "", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_id": "", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_auto_session_clear": "true", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rgwlc_skip_bucket_step": "false", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_block_size": "4096", 2024-05-08T08:25:02.739 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_shard_bits": "4", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_size": "536870912", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cache_type": "binned_lru", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_compaction_stats": "false", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_extended_stats": "false", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_collect_memory_stats": "false", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_index_type": "binary_search", 2024-05-08T08:25:02.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_log_to_ceph_log": "true", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_metadata_block_size": "4096", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_partition_filters": "false", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_perf": "false", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "rotating_keys_renewal_timeout": "10", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "run_dir": "/var/run/ceph", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_block_create": "true", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cache_lru_size": "67108864", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_cbjournal_size": "5368709120", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_data_delta_based_overwrite": "0", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_max_object_size": "16777216", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_device_size": "53687091200", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_capacity": "16", 2024-05-08T08:25:02.741 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_journal_iodepth_limit": "5", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_main_device_type": "SSD", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_max_concurrent_transactions": "8", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "seastore_segment_size": "67108864", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "set_keepcaps": "false", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "setgroup": "", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser": "", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "setuser_match_path": "", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_misplaced_ratio": "0.050000", 2024-05-08T08:25:02.742 INFO:teuthology.orchestra.run.smithi031.stdout: "thp": "false", 2024-05-08T08:25:02.743 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_default_timeout": "60", 2024-05-08T08:25:02.743 INFO:teuthology.orchestra.run.smithi031.stdout: "threadpool_empty_queue_max_wait": "2", 2024-05-08T08:25:02.743 INFO:teuthology.orchestra.run.smithi031.stdout: "throttler_perf_counter": "true" 2024-05-08T08:25:02.743 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:02.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-05-08T08:25:02.862 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-05-08T08:25:02.862 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:25:02.873 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-05-08T08:25:02.873 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:25:02.881 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-05-08T08:25:02.881 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:25:02.892 DEBUG:teuthology.orchestra.run.smithi031:> type ceph-objectstore-tool 2024-05-08T08:25:02.907 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T08:25:02.907 DEBUG:teuthology.orchestra.run.smithi149:> type ceph-objectstore-tool 2024-05-08T08:25:02.925 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T08:25:02.926 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-05-08T08:25:02.936 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-05-08T08:25:02.937 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-05-08T08:25:02.937 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T08:25:02.937 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:25:02.937 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T08:25:02.937 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:25:02.937 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:25:02.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T08:25:02.938 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-05-08T08:25:02.938 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-05-08T08:25:02.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:02.939 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-05-08T08:25:02.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:02.939 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-05-08T08:25:02.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:25:02.940 DEBUG:teuthology.orchestra.run.smithi031:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-05-08T08:25:02.966 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:02.974+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:02.987 INFO:teuthology.orchestra.run.smithi031.stdout:mapped writes DISABLED 2024-05-08T08:25:02.987 INFO:teuthology.orchestra.run.smithi031.stdout:mapped reads DISABLED 2024-05-08T08:25:02.987 INFO:teuthology.orchestra.run.smithi031.stdout:Seed set to 2443 2024-05-08T08:25:03.040 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:03.048+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:03.075 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:03.075 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T08:25:03.075 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:03.076 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:03.077 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:03.077 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:03.077 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:03.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:03.108 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:03.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:03.127 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:03.127 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:03.141 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:03.148+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:03.145 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:03.146 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:03.164 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:03.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:03.183 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:03.183 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:03.201 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:03.202 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:03.220 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:03.220 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:03.222 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:03.222 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:03.232 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:03.238 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:03.239 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:03.241 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:03.249+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:03.342 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:03.349+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:03.348 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:03.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:03.443 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:03.450+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:03.507 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:03.507 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:03.516 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:03.557 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:03.564+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:03.587 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:25:03.646 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:03.653+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:03.672 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:03.672 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:03.681 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:03.748 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:03.755+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:03.790 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:03.791 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:03.800 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:03.848 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:03.856+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:03.909 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:03.910 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:03.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:03.949 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:03.957+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:04.050 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:04.057+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:04.068 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:04.068 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:04.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:04.161 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:04.168+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:04.227 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:04.227 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:04.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:04.251 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:04.259+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:04.352 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:04.360+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:04.386 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:04.386 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:04.395 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:04.454 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:04.462+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:04.545 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:04.545 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:04.554 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:04.555 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:04.563+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:04.657 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:04.665+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:04.672 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:04.672 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:04.681 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:04.758 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:04.766+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:04.795 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:04.795 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:04.804 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:04.859 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:04.867+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:04.911 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:04.911 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:04.919 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:04.960 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:04.968+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:05.061 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:05.069+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:05.065 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:05.065 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:05.074 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:05.162 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:05.170+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:05.179 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:05.179 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:05.188 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:05.251 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:05.295 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:05.302+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:05.334 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:05.334 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:05.342 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:05.364 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:05.371+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:05.390 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:05.390 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:05.408 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:05.408 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:05.425 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:05.425 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:05.442 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:05.442 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:05.448 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:05.448 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:05.457 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:05.459 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:05.459 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:05.465 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:05.472+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:05.477 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:05.477 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:05.495 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:05.496 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:05.513 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:05.513 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:05.564 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:05.564 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:05.564 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:05.564 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:05.565 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:05.573+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:05.573 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:05.605 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:25:05.666 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:05.674+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:05.680 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:05.680 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:05.680 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:05.680 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:05.680 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:05.689 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:05.768 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:05.775+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:05.797 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:05.797 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:05.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:05.797 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [] 2024-05-08T08:25:05.797 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:05.806 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:05.869 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:05.876+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:05.912 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:05.913 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:05.913 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:05.913 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:05.922 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:05.969 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:05.977+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:06.028 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:06.028 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:06.028 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:06.028 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:06.028 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:06.037 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:06.070 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:06.078+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:06.171 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:06.178+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:06.184 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.6247591929999992, 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:25:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:25:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:86 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.721028+0000", 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.4693378119999991, 2024-05-08T08:25:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0068562709999999997, 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 86 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721028+0000", 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721028+0000", 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721028+0000", 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.35e-07 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721030+0000", 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9089999999999998e-06 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721033+0000", 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.411e-06 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721036+0000", 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6510000000000001e-06 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.722679+0000", 2024-05-08T08:25:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016426259999999999 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.722722+0000", 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3614e-05 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.722792+0000", 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9999000000000005e-05 2024-05-08T08:25:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.727263+0000", 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044711860000000003 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.727821+0000", 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00055779199999999999 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.727849+0000", 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7651e-05 2024-05-08T08:25:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.727884+0000", 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5197000000000002e-05 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:87 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.721042+0000", 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.4693236990000003, 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0070468550000000003, 2024-05-08T08:25:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 87 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721042+0000", 2024-05-08T08:25:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721042+0000", 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721042+0000", 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4600000000000001e-07 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721078+0000", 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5376000000000001e-05 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721082+0000", 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721086+0000", 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1350000000000001e-06 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.723002+0000", 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0019161250000000001 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.723047+0000", 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5209e-05 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.723118+0000", 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1581999999999993e-05 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.727293+0000", 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041742610000000003 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728028+0000", 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00073517500000000002 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728054+0000", 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6440000000000001e-05 2024-05-08T08:25:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728089+0000", 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4482e-05 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:88 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.4692737880000006, 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084184770000000006, 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 88 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721094+0000", 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1380000000000001e-06 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721098+0000", 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4120000000000001e-06 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721100+0000", 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6910000000000002e-06 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725525+0000", 2024-05-08T08:25:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044250560000000001 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725656+0000", 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013079000000000001 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725765+0000", 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000108571 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728094+0000", 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023291229999999998 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729456+0000", 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013616780000000001 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729487+0000", 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0956e-05 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729510+0000", 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3711999999999999e-05 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:106 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920040+0000", 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.270325755, 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083085020000000006, 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 106 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920040+0000", 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920040+0000", 2024-05-08T08:25:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920040+0000", 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7000000000000001e-07 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920067+0000", 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6899000000000001e-05 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920072+0000", 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5390000000000001e-06 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920075+0000", 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.128e-06 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920147+0000", 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2026999999999993e-05 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7085999999999997e-05 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920251+0000", 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6703000000000002e-05 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923974+0000", 2024-05-08T08:25:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037232860000000001 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928259+0000", 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042849280000000003 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928317+0000", 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8610000000000003e-05 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928348+0000", 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1025999999999997e-05 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:107 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2702752640000003, 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083640399999999997, 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 107 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920091+0000", 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920093+0000", 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.661e-06 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920098+0000", 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2560000000000004e-06 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920101+0000", 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.932e-06 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920638+0000", 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053755999999999997 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920689+0000", 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0627999999999997e-05 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920762+0000", 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2768999999999994e-05 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925616+0000", 2024-05-08T08:25:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048541820000000003 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928394+0000", 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027782010000000001 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928431+0000", 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6915000000000001e-05 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928454+0000", 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3547999999999999e-05 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2702594939999994, 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084564540000000004, 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 108 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920108+0000", 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9740000000000001e-06 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920112+0000", 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920114+0000", 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6630000000000002e-06 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920996+0000", 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00088141399999999996 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921044+0000", 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8078000000000003e-05 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921120+0000", 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6469000000000003e-05 2024-05-08T08:25:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925621+0000", 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045009630000000002 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928490+0000", 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028691929999999999 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928539+0000", 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8857000000000003e-05 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928563+0000", 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3302000000000002e-05 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2702475070000006, 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085274359999999993, 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 109 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1e-07 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920144+0000", 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.586e-05 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920148+0000", 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1790000000000001e-06 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920151+0000", 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0359999999999998e-06 2024-05-08T08:25:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921404+0000", 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00125264 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921448+0000", 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4227000000000002e-05 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921520+0000", 2024-05-08T08:25:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1532999999999994e-05 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925631+0000", 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041116260000000002 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928593+0000", 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002961881 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928621+0000", 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7259000000000001e-05 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928646+0000", 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4984999999999999e-05 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2702065820000001, 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085697380000000004, 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920162+0000", 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920165+0000", 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4640000000000002e-06 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920168+0000", 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6249999999999999e-06 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921849+0000", 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016816789999999999 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921895+0000", 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5198000000000003e-05 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921968+0000", 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3178999999999998e-05 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925638+0000", 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036703859999999999 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928677+0000", 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003038303 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928708+0000", 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1186000000000003e-05 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928729+0000", 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1035000000000001e-05 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2701937680000004, 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086906299999999995, 2024-05-08T08:25:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 111 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.03e-07 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920174+0000", 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9230000000000001e-06 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920177+0000", 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0539999999999999e-06 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920180+0000", 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8480000000000001e-06 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922200+0000", 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020199060000000001 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922246+0000", 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6319999999999997e-05 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0517000000000003e-05 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925642+0000", 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033256649999999998 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928770+0000", 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031275640000000002 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928829+0000", 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8690000000000002e-05 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928862+0000", 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3939999999999997e-05 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2701819210000007, 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087670669999999999, 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 112 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920214+0000", 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9626999999999999e-05 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920218+0000", 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1320000000000004e-06 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920225+0000", 2024-05-08T08:25:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5859999999999999e-06 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922574+0000", 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002349009 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922651+0000", 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7014999999999994e-05 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922793+0000", 2024-05-08T08:25:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141839 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928736+0000", 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005942363 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928901+0000", 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00016587700000000001 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928930+0000", 2024-05-08T08:25:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8487e-05 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928951+0000", 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0908e-05 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2701314499999992, 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087948200000000001, 2024-05-08T08:25:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 113 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.79e-07 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920237+0000", 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.368e-06 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920241+0000", 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920244+0000", 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7549999999999999e-06 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923291+0000", 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030469809999999998 2024-05-08T08:25:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923338+0000", 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6501000000000001e-05 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923409+0000", 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1198000000000005e-05 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928739+0000", 2024-05-08T08:25:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053299819999999996 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928985+0000", 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00024582599999999999 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929010+0000", 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5145999999999999e-05 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929029+0000", 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9184000000000001e-05 2024-05-08T08:25:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.270116754, 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088585829999999997, 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 114 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.11e-07 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920251+0000", 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0310000000000001e-06 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920254+0000", 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1549999999999999e-06 2024-05-08T08:25:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920257+0000", 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7360000000000001e-06 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923648+0000", 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033905189999999998 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923693+0000", 2024-05-08T08:25:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170999999999998e-05 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923764+0000", 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0875000000000002e-05 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928874+0000", 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051101599999999999 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929058+0000", 2024-05-08T08:25:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018473 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929087+0000", 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8495000000000001e-05 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929107+0000", 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.05e-05 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2701042470000008, 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089168839999999999, 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 115 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920262+0000", 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:25:06.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920301+0000", 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9572e-05 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920305+0000", 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2390000000000004e-06 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920309+0000", 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.14e-06 2024-05-08T08:25:06.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924004+0000", 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036953369999999999 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924048+0000", 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4342999999999999e-05 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924120+0000", 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1489999999999995e-05 2024-05-08T08:25:06.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928881+0000", 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047611650000000004 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929135+0000", 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025369599999999998 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929159+0000", 2024-05-08T08:25:06.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4479999999999999e-05 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929178+0000", 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9208999999999999e-05 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2700516210000004, 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089378329999999992, 2024-05-08T08:25:06.267 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 116 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6199999999999999e-07 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920316+0000", 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1260000000000001e-06 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920326+0000", 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4229999999999994e-06 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920330+0000", 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924512+0000", 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041817440000000003 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924558+0000", 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6285000000000002e-05 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924639+0000", 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0653000000000002e-05 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928884+0000", 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042451420000000004 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929204+0000", 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032002899999999999 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929232+0000", 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8048e-05 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929252+0000", 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9581000000000001e-05 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.272 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2700293229999993, 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008977779, 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 117 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.273 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920337+0000", 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7000000000000001e-07 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920341+0000", 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3039999999999998e-06 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920346+0000", 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920348+0000", 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6809999999999998e-06 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925057+0000", 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047081240000000002 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925101+0000", 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4838e-05 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925174+0000", 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2619000000000004e-05 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928959+0000", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037846730000000001 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929272+0000", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00031341099999999998 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929296+0000", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4226000000000001e-05 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929314+0000", 2024-05-08T08:25:06.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7808999999999998e-05 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:121 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [write 262144~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.929679+0000", 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.2606869140000008, 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0070390610000000001, 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 121 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929679+0000", 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929679+0000", 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929679+0000", 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929718+0000", 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8523999999999999e-05 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929722+0000", 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.391e-06 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929726+0000", 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5810000000000001e-06 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.930858+0000", 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001132168 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.930902+0000", 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4027999999999999e-05 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.931025+0000", 2024-05-08T08:25:06.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012329100000000001 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.934419+0000", 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033943799999999998 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.936661+0000", 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022412650000000001 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.936693+0000", 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2042999999999998e-05 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.936718+0000", 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5046999999999999e-05 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 7.5740373070000002, 2024-05-08T08:25:06.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083370549999999995, 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 139 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616329+0000", 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0499999999999999e-07 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616332+0000", 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8219999999999998e-06 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616336+0000", 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616339+0000", 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1650000000000002e-06 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621594+0000", 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052553139999999996 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621649+0000", 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4416000000000002e-05 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621721+0000", 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1977000000000006e-05 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.623088+0000", 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013677889999999999 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624604+0000", 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0015159279999999999 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624643+0000", 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8504999999999998e-05 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624665+0000", 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2543000000000001e-05 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 7.5740196730000005, 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083691970000000001, 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 140 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.41e-07 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616374+0000", 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8e-05 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616379+0000", 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3579999999999996e-06 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616381+0000", 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8389999999999998e-06 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621985+0000", 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056040489999999998 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622031+0000", 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5683000000000003e-05 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622098+0000", 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7063000000000005e-05 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.623381+0000", 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001283228 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624673+0000", 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001291295 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624697+0000", 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4122999999999999e-05 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624715+0000", 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8318e-05 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 7.5739775639999998, 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098144059999999995, 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:06.294 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 141 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3299999999999998e-07 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616391+0000", 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4360000000000001e-06 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616395+0000", 2024-05-08T08:25:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.822e-06 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616397+0000", 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.729e-06 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622404+0000", 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0060065120000000003 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:06.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622448+0000", 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3707e-05 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622520+0000", 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624923+0000", 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024033819999999999 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626156+0000", 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001232216 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626183+0000", 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7347000000000001e-05 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626202+0000", 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9564000000000001e-05 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.299 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.300 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.300 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:06.300 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:06.300 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:06.301 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:06.309+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:06.301 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:06.402 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:06.409+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:06.419 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:06.419 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:06.419 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:06.428 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:06.512 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:06.519+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:06.577 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:06.578 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:06.578 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:06.578 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:06.578 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:06.586 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:06.599 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:25:06.616 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:06.623+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:06.704 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:06.711+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:06.713 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:06.714 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [] 2024-05-08T08:25:06.714 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:06.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:06.807 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:06.815+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:06.874 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:06.874 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:06.875 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:06.883 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:06.925 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:06.932+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:07.007 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:07.015+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:07.033 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:07.033 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:07.034 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:07.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:07.108 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:07.116+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:07.154 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:07.154 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:07.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:07.154 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:1 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.5392760780000003, 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075841290000000002, 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621510+0000", 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9610000000000002e-06 2024-05-08T08:25:07.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621517+0000", 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0949999999999998e-06 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621526+0000", 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621572+0000", 2024-05-08T08:25:07.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5599000000000002e-05 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621600+0000", 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8399e-05 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.629024+0000", 2024-05-08T08:25:07.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074240119999999998 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.629088+0000", 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3455999999999997e-05 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:83 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4390912690000004, 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052916229999999996, 2024-05-08T08:25:07.159 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721689+0000", 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721697+0000", 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721706+0000", 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0000000000000002e-06 2024-05-08T08:25:07.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721713+0000", 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2380000000000002e-06 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721822+0000", 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010911 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721845+0000", 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3833999999999999e-05 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726930+0000", 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050846830000000004 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726980+0000", 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9957999999999998e-05 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:84 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4386238339999995, 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048602560000000003, 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722157+0000", 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7899999999999998e-07 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722191+0000", 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4762999999999999e-05 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722201+0000", 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2380000000000003e-06 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722209+0000", 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8019999999999998e-06 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722251+0000", 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1128999999999999e-05 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722268+0000", 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7371e-05 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726987+0000", 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047187699999999997 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727016+0000", 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9504000000000001e-05 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.167 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:85 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'55, mlcod=16'50)", 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4381958509999997, 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044628130000000004, 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000003e-07 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722593+0000", 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722602+0000", 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0920000000000004e-06 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722609+0000", 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722733+0000", 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012490099999999999 2024-05-08T08:25:07.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722756+0000", 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2819e-05 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727018+0000", 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042616590000000001 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727047+0000", 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8796e-05 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.171 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:86 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'50)", 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.722926+0000", 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4378541840000008, 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041511789999999996, 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722926+0000", 2024-05-08T08:25:07.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722926+0000", 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722926+0000", 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7599999999999997e-07 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722931+0000", 2024-05-08T08:25:07.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722941+0000", 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2199999999999998e-06 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722947+0000", 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1290000000000001e-06 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.723012+0000", 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5650999999999999e-05 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.723033+0000", 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0835e-05 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727049+0000", 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00401535 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727077+0000", 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8153e-05 2024-05-08T08:25:07.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:104 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.919836+0000", 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2409434689999994, 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042493720000000004, 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919836+0000", 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919836+0000", 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919838+0000", 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1610000000000001e-06 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919851+0000", 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919879+0000", 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.756e-05 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919896+0000", 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6600999999999999e-05 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919971+0000", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5301000000000003e-05 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919997+0000", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5851e-05 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924043+0000", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040459479999999997 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924086+0000", 2024-05-08T08:25:07.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3022000000000002e-05 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2401266389999996, 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073213000000000002, 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920654+0000", 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920686+0000", 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2338000000000001e-05 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920695+0000", 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5650000000000007e-06 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920701+0000", 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8170000000000004e-06 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920749+0000", 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8071000000000001e-05 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920768+0000", 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8546000000000002e-05 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927915+0000", 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071468269999999997 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927974+0000", 2024-05-08T08:25:07.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9648000000000002e-05 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2396000170000008, 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068250300000000002, 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-07 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921185+0000", 2024-05-08T08:25:07.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.135e-06 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921195+0000", 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921201+0000", 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921321+0000", 2024-05-08T08:25:07.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011977 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921339+0000", 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7608000000000001e-05 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927977+0000", 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066382029999999996 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928005+0000", 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7722e-05 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.187 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2395324910000003, 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067827720000000003, 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921248+0000", 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921252+0000", 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4449999999999998e-06 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921261+0000", 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5830000000000007e-06 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921267+0000", 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921577+0000", 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031014399999999999 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921595+0000", 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7802e-05 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928006+0000", 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064114100000000002 2024-05-08T08:25:07.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928030+0000", 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4042000000000001e-05 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:109 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'64)", 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2389538499999997, 2024-05-08T08:25:07.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062286720000000002, 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.44e-07 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921855+0000", 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8645000000000001e-05 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921865+0000", 2024-05-08T08:25:07.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7310000000000005e-06 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921871+0000", 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921924+0000", 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3109999999999998e-05 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921945+0000", 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1118e-05 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928031+0000", 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060861200000000004 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928055+0000", 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3141000000000001e-05 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.196 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:110 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'64)", 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2384818129999999, 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057807739999999998, 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922304+0000", 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:07.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.288e-05 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922323+0000", 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3749999999999999e-06 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922387+0000", 2024-05-08T08:25:07.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3932999999999999e-05 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922407+0000", 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9885e-05 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928056+0000", 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056492180000000001 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928079+0000", 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2467e-05 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:111 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'64)", 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2381359980000006, 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005461502, 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-05-08T08:25:07.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922649+0000", 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922658+0000", 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8419999999999994e-06 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922664+0000", 2024-05-08T08:25:07.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1290000000000001e-06 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922725+0000", 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922743+0000", 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8142999999999999e-05 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928080+0000", 2024-05-08T08:25:07.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053372430000000002 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928105+0000", 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5004999999999999e-05 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'64)", 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2378186860000007, 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055842340000000004, 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.205 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922962+0000", 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923087+0000", 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012524199999999999 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923097+0000", 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8700000000000004e-06 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923103+0000", 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2049999999999999e-06 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923163+0000", 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0554000000000003e-05 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923182+0000", 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8515000000000001e-05 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928508+0000", 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053261330000000003 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928545+0000", 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7165000000000001e-05 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:113 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'64)", 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2372408939999993, 2024-05-08T08:25:07.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050342659999999999, 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0200000000000002e-07 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923547+0000", 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2119999999999997e-06 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923556+0000", 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3470000000000005e-06 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923564+0000", 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8779999999999998e-06 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923625+0000", 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1309999999999994e-05 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923644+0000", 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8403999999999999e-05 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928547+0000", 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049035349999999997 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928573+0000", 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5978000000000001e-05 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:114 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'76, mlcod=16'64)", 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.923916+0000", 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.236863585, 2024-05-08T08:25:07.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046851849999999997, 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923916+0000", 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923916+0000", 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923917+0000", 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6400000000000002e-07 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923922+0000", 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923934+0000", 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1206000000000001e-05 2024-05-08T08:25:07.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923940+0000", 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.088e-06 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924190+0000", 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025044899999999997 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924208+0000", 2024-05-08T08:25:07.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8008000000000001e-05 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928577+0000", 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043690730000000002 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928601+0000", 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4207e-05 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:115 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'64)", 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2364311319999999, 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042775119999999998, 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5499999999999997e-07 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924418+0000", 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8512999999999999e-05 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924430+0000", 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2037e-05 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924437+0000", 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3019999999999997e-06 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924482+0000", 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4697999999999998e-05 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924500+0000", 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7666999999999999e-05 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928603+0000", 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041028770000000004 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928626+0000", 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3563000000000001e-05 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:116 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'78, mlcod=16'64)", 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.924776+0000", 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2360040100000003, 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00387441, 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924776+0000", 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924776+0000", 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924776+0000", 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8199999999999999e-07 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924782+0000", 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5149999999999997e-06 2024-05-08T08:25:07.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924793+0000", 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1066e-05 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924800+0000", 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6290000000000004e-06 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924836+0000", 2024-05-08T08:25:07.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6127000000000002e-05 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924853+0000", 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7618999999999999e-05 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928627+0000", 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003773715 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928650+0000", 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3057e-05 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.929954+0000", 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2308253069999999, 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040244160000000003, 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.226 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.929954+0000", 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.929954+0000", 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.929956+0000", 2024-05-08T08:25:07.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.378e-06 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.929992+0000", 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6115999999999998e-05 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.930013+0000", 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0636999999999999e-05 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.930025+0000", 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2840000000000001e-05 2024-05-08T08:25:07.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.930110+0000", 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4464999999999999e-05 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.930133+0000", 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3493e-05 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.933932+0000", 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037983370000000002 2024-05-08T08:25:07.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.933979+0000", 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7150000000000001e-05 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:143 2.0 e17/15 2:30306672:devicehealth::main.db.0000000000000000:head v 17'101, mlcod=17'98)", 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:58.627220+0000", 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5335598340000001, 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004400101, 2024-05-08T08:25:07.230 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627220+0000", 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627220+0000", 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627221+0000", 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-07 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627283+0000", 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2219999999999997e-05 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627294+0000", 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1521e-05 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627300+0000", 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627342+0000", 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2255000000000001e-05 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627355+0000", 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.631580+0000", 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042257320000000003 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.631620+0000", 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9739e-05 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:144 2.0 e17/15 2:30306672:devicehealth::main.db.0000000000000000:head v 17'102, mlcod=17'98)", 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:58.627628+0000", 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.5331515660000008, 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040091140000000003, 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627628+0000", 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627628+0000", 2024-05-08T08:25:07.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627629+0000", 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4300000000000002e-07 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627700+0000", 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0617999999999994e-05 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627712+0000", 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1979e-05 2024-05-08T08:25:07.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627725+0000", 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3559e-05 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627767+0000", 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1510000000000001e-05 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627780+0000", 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:25:07.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.631622+0000", 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038422069999999998 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.631637+0000", 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.506e-05 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:07.238 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:07.240 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:07.241 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:07.248+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:07.341 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:07.348+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:07.350 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:07.350 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:07.350 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:07.350 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:07.359 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:07.442 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:07.450+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:07.510 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:07.511 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:07.511 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:07.519 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:07.524 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:07.544 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:07.551+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:07.631 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:07.631 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:07.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:07.631 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [] 2024-05-08T08:25:07.631 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:07.641 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:07.684 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:07.692+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:07.692 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:07.692 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:07.710 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:07.710 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:07.727 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:07.728 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:07.745 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:07.745 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:07.745 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:07.752+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:07.761 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:07.761 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:07.774 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:07.778 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:07.778 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:07.796 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:07.797 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:07.815 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:07.815 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:07.833 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:07.833 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:07.845 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:07.853+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:07.889 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:07.889 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:07.898 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:07.946 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:07.953+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:08.009 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:08.009 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:08.018 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:08.063 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:08.071+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:08.088 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T08:25:08.088 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:25:08.088 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:25:08.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:25:08.134 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:08.135 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:08.144 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:08.147 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:08.155+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:08.248 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:08.256+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:08.291 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:08.291 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:08.300 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:08.349 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:08.357+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:08.351 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:08.352 INFO:teuthology.orchestra.run.smithi031.stdout:skipping zero size read 2024-05-08T08:25:08.352 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:08.369 INFO:teuthology.orchestra.run.smithi031.stdout:6 writesame 0x9c09d3d thru 0x9c0f81a data_size 0x3(0x5ade bytes) 2024-05-08T08:25:08.380 INFO:teuthology.orchestra.run.smithi031.stdout:7 write 0xe92ad78 thru 0xe934248 (0x94d1 bytes) 2024-05-08T08:25:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:10 read 0x93f4daf thru 0x940459e (0xf7f0 bytes) 2024-05-08T08:25:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xe6d26a8 2024-05-08T08:25:08.383 INFO:teuthology.orchestra.run.smithi031.stdout:11 trunc from 0xe934249 to 0xe6d26a8 2024-05-08T08:25:08.393 INFO:teuthology.orchestra.run.smithi031.stdout:12 writesame 0x42c0233 thru 0x42cfcaf data_size 0x5f(0xfa7d bytes) 2024-05-08T08:25:08.397 INFO:teuthology.orchestra.run.smithi031.stdout:14 read 0xe0b284 thru 0xe14de1 (0x9b5e bytes) 2024-05-08T08:25:08.397 INFO:teuthology.orchestra.run.smithi031.stdout:15 read 0xeb710b thru 0xec4422 (0xd318 bytes) 2024-05-08T08:25:08.397 INFO:teuthology.orchestra.run.smithi031.stdout:16 write 0xa4de830 thru 0xa4e88fd (0xa0ce bytes) 2024-05-08T08:25:08.403 INFO:teuthology.orchestra.run.smithi031.stdout:17 punch from 0xc49e913 to 0xc4a2980, (0x406d bytes) 2024-05-08T08:25:08.403 INFO:teuthology.orchestra.run.smithi031.stdout:19 write 0x8293922 thru 0x829b0b4 (0x7793 bytes) 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 21, 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf", 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-05-08T08:24:44.143941+0000", 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-05-08T08:25:06.604296+0000", 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-05-08T08:24:54.547557+0000", 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-05-08T08:24:46.429925+0000", 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-05-08T08:25:08.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:08.405 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-05-08T08:25:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:25:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:25:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:25:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-05-08T08:25:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-05-08T08:25:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-05-08T08:25:08.406 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:25:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-05-08T08:25:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-05-08T08:25:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-05-08T08:25:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-05-08T08:25:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-05-08T08:25:08.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-05-08T08:25:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:55.134472+0000", 2024-05-08T08:25:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:25:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:25:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:25:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:25:08.408 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:25:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:25:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:25:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:25:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:25:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:25:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:25:08.409 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:25:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-05-08T08:25:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-05-08T08:25:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-05-08T08:25:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-05-08T08:25:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-05-08T08:25:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:25:08.410 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "16", 2024-05-08T08:25:08.411 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:25:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:25:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:25:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:25:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:25:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:25:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:25:08.412 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:25:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:25:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:25:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:25:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:25:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:25:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:25:08.413 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:25:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:25:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:25:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:25:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:25:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:25:08.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:25:08.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2, 2024-05-08T08:25:08.416 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:56.465515+0000", 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:25:08.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-05-08T08:25:08.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:25:08.419 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:25:08.420 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:25:08.421 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 8, 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 8, 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.25, 2024-05-08T08:25:08.422 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:25:03.054989+0000", 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:25:08.423 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:25:08.424 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "21", 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 2, 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 21, 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:25:08.425 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:25:08.426 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:25:08.427 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.5, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.5, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-05-08T08:25:08.428 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.5, 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.5, 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:25:08.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14afea42-356a-4163-ac3f-2e270e807917", 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.430 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.431 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-05-08T08:25:08.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/727789107", 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/727789107", 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/727789107", 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/727789107", 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "db77a923-8f49-4b47-b752-00b475917a0b", 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.436 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/3462314176", 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/3462314176", 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/3462314176", 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/3462314176", 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "3d508691-8e40-4592-82d0-eaf4c4c7d7dd", 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 0, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.442 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.444 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:08.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2736372115", 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2736372115", 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2736372115", 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2736372115", 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:25:08.447 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2545300a-428a-41ed-b19a-3a0cb9e5e755", 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.448 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/573482428", 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/573482428", 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/573482428", 2024-05-08T08:25:08.452 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/573482428", 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "886aeed5-4495-4064-8579-050d38a4ad1b", 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.453 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6801", 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.454 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6803", 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6805", 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6807", 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6813", 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6815", 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6809", 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6811", 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.457 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6803/3424463091", 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6807/3424463091", 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6815/3424463091", 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6811/3424463091", 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.458 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4d15cef0-9ee9-4069-a6c4-a650e873ee03", 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6816", 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6818", 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6820", 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6822", 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6828", 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6830", 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6824", 2024-05-08T08:25:08.462 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6826", 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6818/3709461555", 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6822/3709461555", 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6830/3709461555", 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6826/3709461555", 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.463 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "910ac664-9573-4851-9e2b-89ac6015fdc1", 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:08.464 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6800", 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6802", 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6804", 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6806", 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6812", 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6814", 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6808", 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6810", 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6802/405561966", 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6806/405561966", 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6814/405561966", 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6810/405561966", 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14cc2d94-48e4-47c9-896a-9be9281ef0c5", 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:08.469 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6817", 2024-05-08T08:25:08.470 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6819", 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6821", 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6823", 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6829", 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6831", 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6825", 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6827", 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6819/570236217", 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6823/570236217", 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6831/570236217", 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6827/570236217", 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:08.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.572981+0000", 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.626543+0000", 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.476 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.638878+0000", 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.599293+0000", 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.565676+0000", 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:25:08.478 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.552124+0000", 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.479 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.531428+0000", 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.658391+0000", 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:08.480 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-05-08T08:25:08.481 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 2, 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.482 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 2, 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-05-08T08:25:08.483 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-05-08T08:25:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-05-08T08:25:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-05-08T08:25:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:25:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:25:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-05-08T08:25:08.484 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:08.484 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:08.485 INFO:teuthology.orchestra.run.smithi031.stdout:21 write 0xaf25fc5 thru 0xaf31586 (0xb5c2 bytes) 2024-05-08T08:25:08.485 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:08.486 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:08.486 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:08.487 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:08.488 INFO:tasks.thrashosds.thrasher:j is {'epoch': 21, 'fsid': '1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf', 'created': '2024-05-08T08:24:44.143941+0000', 'modified': '2024-05-08T08:25:06.604296+0000', 'last_up_change': '2024-05-08T08:24:54.547557+0000', 'last_in_change': '2024-05-08T08:24:46.429925+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:24:55.134472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:24:56.465515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:25:03.054989+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 21, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '14afea42-356a-4163-ac3f-2e270e807917', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 727789107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 727789107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 727789107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 727789107}]}, 'public_addr': '172.21.15.31:6801/727789107', 'cluster_addr': '172.21.15.31:6803/727789107', 'heartbeat_back_addr': '172.21.15.31:6807/727789107', 'heartbeat_front_addr': '172.21.15.31:6805/727789107', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'db77a923-8f49-4b47-b752-00b475917a0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 3462314176}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 3462314176}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 3462314176}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 3462314176}]}, 'public_addr': '172.21.15.31:6817/3462314176', 'cluster_addr': '172.21.15.31:6819/3462314176', 'heartbeat_back_addr': '172.21.15.31:6823/3462314176', 'heartbeat_front_addr': '172.21.15.31:6821/3462314176', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '3d508691-8e40-4592-82d0-eaf4c4c7d7dd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2736372115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2736372115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2736372115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2736372115}]}, 'public_addr': '172.21.15.31:6809/2736372115', 'cluster_addr': '172.21.15.31:6811/2736372115', 'heartbeat_back_addr': '172.21.15.31:6815/2736372115', 'heartbeat_front_addr': '172.21.15.31:6813/2736372115', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2545300a-428a-41ed-b19a-3a0cb9e5e755', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 573482428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 573482428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 573482428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 573482428}]}, 'public_addr': '172.21.15.31:6825/573482428', 'cluster_addr': '172.21.15.31:6827/573482428', 'heartbeat_back_addr': '172.21.15.31:6831/573482428', 'heartbeat_front_addr': '172.21.15.31:6829/573482428', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '886aeed5-4495-4064-8579-050d38a4ad1b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6801', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6803', 'nonce': 3424463091}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6805', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6807', 'nonce': 3424463091}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6813', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6815', 'nonce': 3424463091}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6809', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6811', 'nonce': 3424463091}]}, 'public_addr': '172.21.15.149:6803/3424463091', 'cluster_addr': '172.21.15.149:6807/3424463091', 'heartbeat_back_addr': '172.21.15.149:6815/3424463091', 'heartbeat_front_addr': '172.21.15.149:6811/3424463091', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d15cef0-9ee9-4069-a6c4-a650e873ee03', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6816', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6818', 'nonce': 3709461555}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6820', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6822', 'nonce': 3709461555}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6828', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6830', 'nonce': 3709461555}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6824', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6826', 'nonce': 3709461555}]}, 'public_addr': '172.21.15.149:6818/3709461555', 'cluster_addr': '172.21.15.149:6822/3709461555', 'heartbeat_back_addr': '172.21.15.149:6830/3709461555', 'heartbeat_front_addr': '172.21.15.149:6826/3709461555', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '910ac664-9573-4851-9e2b-89ac6015fdc1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6800', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6802', 'nonce': 405561966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6804', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6806', 'nonce': 405561966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6812', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6814', 'nonce': 405561966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6808', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6810', 'nonce': 405561966}]}, 'public_addr': '172.21.15.149:6802/405561966', 'cluster_addr': '172.21.15.149:6806/405561966', 'heartbeat_back_addr': '172.21.15.149:6814/405561966', 'heartbeat_front_addr': '172.21.15.149:6810/405561966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '14cc2d94-48e4-47c9-896a-9be9281ef0c5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6817', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6819', 'nonce': 570236217}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6821', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6823', 'nonce': 570236217}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6829', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6831', 'nonce': 570236217}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6825', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6827', 'nonce': 570236217}]}, 'public_addr': '172.21.15.149:6819/570236217', 'cluster_addr': '172.21.15.149:6823/570236217', 'heartbeat_back_addr': '172.21.15.149:6831/570236217', 'heartbeat_front_addr': '172.21.15.149:6827/570236217', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.572981+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.626543+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.638878+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.599293+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.565676+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.552124+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.531428+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.658391+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 2, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:25:08.488 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-05-08T08:25:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:24 trunc from 0xe6d26a8 to 0xd963e4 2024-05-08T08:25:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:25 read 0x39dc9 thru 0x495a8 (0xf7e0 bytes) 2024-05-08T08:25:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:28 trunc from 0xd963e4 to 0x8412d5e 2024-05-08T08:25:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:29 read 0x3be1524 thru 0x3be9b05 (0x85e2 bytes) 2024-05-08T08:25:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:30 write 0x61da7f4 thru 0x61de785 (0x3f92 bytes) 2024-05-08T08:25:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:32 read 0x17047bf thru 0x170ba8a (0x72cc bytes) 2024-05-08T08:25:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:33 clone 1 order 18 su 2048 sc 6 2024-05-08T08:25:08.490 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:08.493+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:08.586 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:08.594+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:08.595 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:08.595 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:08.605 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:08.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:25:08.689 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:08.697+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:08.756 INFO:teuthology.orchestra.run.smithi031.stdout:34 punch from 0x104dcee to 0x1051285, (0x3597 bytes) 2024-05-08T08:25:08.756 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:08.757 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:08.766 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:08.767 INFO:teuthology.orchestra.run.smithi031.stdout:35 trunc from 0x8412d5e to 0x1abfe10 2024-05-08T08:25:08.788 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:08.795+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:08.792 INFO:teuthology.orchestra.run.smithi031.stdout:36 write 0x6107573 thru 0x61156eb (0xe179 bytes) 2024-05-08T08:25:08.802 INFO:teuthology.orchestra.run.smithi031.stdout:37 read 0x31c7c28 thru 0x31cb007 (0x33e0 bytes) 2024-05-08T08:25:08.803 INFO:teuthology.orchestra.run.smithi031.stdout:39 punch from 0x4397532 to 0x439ecbb, (0x7789 bytes) 2024-05-08T08:25:08.803 INFO:teuthology.orchestra.run.smithi031.stdout:40 write 0x11abffb thru 0x11b3c4d (0x7c53 bytes) 2024-05-08T08:25:08.820 INFO:teuthology.orchestra.run.smithi031.stdout:41 writesame 0x8015089 thru 0x801fab9 data_size 0x2f(0xaa31 bytes) 2024-05-08T08:25:08.820 INFO:teuthology.orchestra.run.smithi031.stdout:42 read 0x3f20b08 thru 0x3f227b7 (0x1cb0 bytes) 2024-05-08T08:25:08.820 INFO:teuthology.orchestra.run.smithi031.stdout:44 read 0x632651 thru 0x632e3f (0x7ef bytes) 2024-05-08T08:25:08.820 INFO:teuthology.orchestra.run.smithi031.stdout:48 punch from 0x1a758d to 0x1a8ccc, (0x173f bytes) 2024-05-08T08:25:08.828 INFO:teuthology.orchestra.run.smithi031.stdout:49 writesame 0xa5ebe3b thru 0xa5f3894 data_size 0x1(0x7a5a bytes) 2024-05-08T08:25:08.828 INFO:teuthology.orchestra.run.smithi031.stdout:50 read 0x2b896ba thru 0x2b97993 (0xe2da bytes) 2024-05-08T08:25:08.828 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:08.828 INFO:teuthology.orchestra.run.smithi031.stdout:53 read 0x93f5b8f thru 0x9404aaf (0xef21 bytes) 2024-05-08T08:25:08.828 INFO:teuthology.orchestra.run.smithi031.stdout:54 write 0xfa02c6 thru 0xfa4f48 (0x4c83 bytes) 2024-05-08T08:25:08.841 INFO:teuthology.orchestra.run.smithi031.stdout:55 write 0x6195617 thru 0x619fad0 (0xa4ba bytes) 2024-05-08T08:25:08.854 INFO:teuthology.orchestra.run.smithi031.stdout:56 write 0x63301f5 thru 0x633a39a (0xa1a6 bytes) 2024-05-08T08:25:08.866 INFO:teuthology.orchestra.run.smithi031.stdout:57 read 0x9da9734 thru 0x9db31a6 (0x9a73 bytes) 2024-05-08T08:25:08.867 INFO:teuthology.orchestra.run.smithi031.stdout:58 read 0x37c5a08 thru 0x37ca211 (0x480a bytes) 2024-05-08T08:25:08.873 INFO:teuthology.orchestra.run.smithi031.stdout:59 write 0xb6224d9 thru 0xb62d3cf (0xaef7 bytes) 2024-05-08T08:25:08.888 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:08.896+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:08.889 INFO:teuthology.orchestra.run.smithi031.stdout:62 writesame 0x2c15595 thru 0x2c162d7 data_size 0x7(0xd43 bytes) 2024-05-08T08:25:08.896 INFO:teuthology.orchestra.run.smithi031.stdout:64 read 0x9468ff6 thru 0x9470288 (0x7293 bytes) 2024-05-08T08:25:08.896 INFO:teuthology.orchestra.run.smithi031.stdout:65 read 0x1c2ace2 thru 0x1c389ce (0xdced bytes) 2024-05-08T08:25:08.902 INFO:teuthology.orchestra.run.smithi031.stdout:66 write 0xb9d6893 thru 0xb9e3b07 (0xd275 bytes) 2024-05-08T08:25:08.919 INFO:teuthology.orchestra.run.smithi031.stdout:67 trunc from 0xb9e3b08 to 0x85f0608 2024-05-08T08:25:08.924 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:08.924 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:08.934 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:08.946 INFO:teuthology.orchestra.run.smithi031.stdout:68 write 0xbd83720 thru 0xbd84367 (0xc48 bytes) 2024-05-08T08:25:08.955 INFO:teuthology.orchestra.run.smithi031.stdout:71 punch from 0xb70a9e2 to 0xb7161a3, (0xb7c1 bytes) 2024-05-08T08:25:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:72 punch from 0x78be9fe to 0x78caa31, (0xc033 bytes) 2024-05-08T08:25:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:73 read 0x4a5671d thru 0x4a60a8f (0xa373 bytes) 2024-05-08T08:25:08.956 INFO:teuthology.orchestra.run.smithi031.stdout:74 write 0xaf290b0 thru 0xaf33020 (0x9f71 bytes) 2024-05-08T08:25:08.968 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:08.968 INFO:teuthology.orchestra.run.smithi031.stdout:76 read 0xb647ec thru 0xb66afc (0x2311 bytes) 2024-05-08T08:25:08.968 INFO:teuthology.orchestra.run.smithi031.stdout:77 read 0x96b8d6d thru 0x96bd08d (0x4321 bytes) 2024-05-08T08:25:08.968 INFO:teuthology.orchestra.run.smithi031.stdout:78 read 0x30842da thru 0x309360c (0xf333 bytes) 2024-05-08T08:25:08.968 INFO:teuthology.orchestra.run.smithi031.stdout:81 write 0x7af9c03 thru 0x7b07a45 (0xde43 bytes) 2024-05-08T08:25:08.982 INFO:teuthology.orchestra.run.smithi031.stdout:82 read 0x487b41 thru 0x493ee2 (0xc3a2 bytes) 2024-05-08T08:25:08.982 INFO:teuthology.orchestra.run.smithi031.stdout:83 write 0x7f8e964 thru 0x7f8f9a8 (0x1045 bytes) 2024-05-08T08:25:08.986 INFO:teuthology.orchestra.run.smithi031.stdout:84 write 0x57fd123 thru 0x5804b1d (0x79fb bytes) 2024-05-08T08:25:08.989 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:08.997+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:08.997 INFO:teuthology.orchestra.run.smithi031.stdout:86 punch from 0x4862654 to 0x48628d7, (0x283 bytes) 2024-05-08T08:25:08.997 INFO:teuthology.orchestra.run.smithi031.stdout:88 write 0xa783930 thru 0xa7895e2 (0x5cb3 bytes) 2024-05-08T08:25:09.007 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.019 INFO:teuthology.orchestra.run.smithi031.stdout:91 trunc from 0xbd84368 to 0xd40e55 2024-05-08T08:25:09.056 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:09.056 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:09.065 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:09.086 INFO:teuthology.orchestra.run.smithi031.stdout:93 write 0xd43c557 thru 0xd4493fd (0xcea7 bytes) 2024-05-08T08:25:09.090 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:09.098+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:09.101 INFO:teuthology.orchestra.run.smithi031.stdout:94 read 0x35f8ab1 thru 0x3601199 (0x86e9 bytes) 2024-05-08T08:25:09.101 INFO:teuthology.orchestra.run.smithi031.stdout:95 punch from 0x482e69d to 0x482fb5a, (0x14bd bytes) 2024-05-08T08:25:09.102 INFO:teuthology.orchestra.run.smithi031.stdout:97 write 0xad407eb thru 0xad46838 (0x604e bytes) 2024-05-08T08:25:09.120 INFO:teuthology.orchestra.run.smithi031.stdout:98 write 0xd52ba75 thru 0xd5315c0 (0x5b4c bytes) 2024-05-08T08:25:09.128 INFO:teuthology.orchestra.run.smithi031.stdout:99 write 0xb4d493b thru 0xb4d80d1 (0x3797 bytes) 2024-05-08T08:25:09.138 INFO:teuthology.orchestra.run.smithi031.stdout:102 read 0x6620826 thru 0x662cb38 (0xc313 bytes) 2024-05-08T08:25:09.138 INFO:teuthology.orchestra.run.smithi031.stdout:103 punch from 0xb184495 to 0xb18c367, (0x7ed2 bytes) 2024-05-08T08:25:09.138 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.138 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.142 INFO:teuthology.orchestra.run.smithi031.stdout:106 trunc from 0xd5315c1 to 0x9782e61 2024-05-08T08:25:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:107 read 0x630212d thru 0x630cb79 (0xaa4d bytes) 2024-05-08T08:25:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:108 read 0x4519cd thru 0x458481 (0x6ab5 bytes) 2024-05-08T08:25:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.166 INFO:teuthology.orchestra.run.smithi031.stdout:110 write 0x1e9eecc thru 0x1ea5e61 (0x6f96 bytes) 2024-05-08T08:25:09.178 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.178 INFO:teuthology.orchestra.run.smithi031.stdout:112 write 0x8316073 thru 0x831717d (0x110b bytes) 2024-05-08T08:25:09.183 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.186 INFO:teuthology.orchestra.run.smithi031.stdout:115 trunc from 0x9782e61 to 0x6eda091 2024-05-08T08:25:09.194 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:09.202+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:09.201 INFO:teuthology.orchestra.run.smithi031.stdout:116 punch from 0x4a08473 to 0x4a12f9c, (0xab29 bytes) 2024-05-08T08:25:09.208 INFO:teuthology.orchestra.run.smithi031.stdout:117 trunc from 0x6eda091 to 0x1270ce8 2024-05-08T08:25:09.222 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:09.222 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:09.228 INFO:teuthology.orchestra.run.smithi031.stdout:118 trunc from 0x1270ce8 to 0x9d00ffc 2024-05-08T08:25:09.232 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:09.235 INFO:teuthology.orchestra.run.smithi031.stdout:120 read 0x6d92990 thru 0x6d92e91 (0x502 bytes) 2024-05-08T08:25:09.235 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.237 INFO:teuthology.orchestra.run.smithi031.stdout:122 trunc from 0x9d00ffc to 0x83f618d 2024-05-08T08:25:09.242 INFO:teuthology.orchestra.run.smithi031.stdout:123 read 0x43494eb thru 0x434e882 (0x5398 bytes) 2024-05-08T08:25:09.242 INFO:teuthology.orchestra.run.smithi031.stdout:124 write 0x783b3b1 thru 0x78479ed (0xc63d bytes) 2024-05-08T08:25:09.249 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xe9932eb 2024-05-08T08:25:09.256 INFO:teuthology.orchestra.run.smithi031.stdout:126 trunc from 0x83f618d to 0xe9932eb 2024-05-08T08:25:09.261 INFO:teuthology.orchestra.run.smithi031.stdout:127 read 0xc08c451 thru 0xc099ea3 (0xda53 bytes) 2024-05-08T08:25:09.261 INFO:teuthology.orchestra.run.smithi031.stdout:128 write 0x9045b71 thru 0x904da3d (0x7ecd bytes) 2024-05-08T08:25:09.268 INFO:teuthology.orchestra.run.smithi031.stdout:129 punch from 0x3b56068 to 0x3b56f24, (0xebc bytes) 2024-05-08T08:25:09.268 INFO:teuthology.orchestra.run.smithi031.stdout:130 writesame 0xe0061fd thru 0xe01057c data_size 0x60(0xa380 bytes) 2024-05-08T08:25:09.278 INFO:teuthology.orchestra.run.smithi031.stdout:133 write 0x22ffce5 thru 0x230f86f (0xfb8b bytes) 2024-05-08T08:25:09.286 INFO:teuthology.orchestra.run.smithi031.stdout:134 write 0xc42aa9c thru 0xc42ae16 (0x37b bytes) 2024-05-08T08:25:09.289 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.289 INFO:teuthology.orchestra.run.smithi031.stdout:136 write 0x58fb9be thru 0x5901a40 (0x6083 bytes) 2024-05-08T08:25:09.295 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:09.302+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:09.295 INFO:teuthology.orchestra.run.smithi031.stdout:137 write 0xaffe063 thru 0xb00824c (0xa1ea bytes) 2024-05-08T08:25:09.312 INFO:teuthology.orchestra.run.smithi031.stdout:138 trunc from 0xe9932eb to 0x7a9b9bc 2024-05-08T08:25:09.329 INFO:teuthology.orchestra.run.smithi031.stdout:139 read 0x1942244 thru 0x1947fa8 (0x5d65 bytes) 2024-05-08T08:25:09.338 INFO:teuthology.orchestra.run.smithi031.stdout:141 writesame 0x998f782 thru 0x99985c9 data_size 0x74(0x8e48 bytes) 2024-05-08T08:25:09.344 INFO:teuthology.orchestra.run.smithi031.stdout:142 writesame 0x9ef5a1c thru 0x9ef8639 data_size 0x2(0x2c1e bytes) 2024-05-08T08:25:09.344 INFO:teuthology.orchestra.run.smithi031.stdout:143 write 0x23cd34c thru 0x23da6e5 (0xd39a bytes) 2024-05-08T08:25:09.352 INFO:teuthology.orchestra.run.smithi031.stdout:145 punch from 0x8ad704a to 0x8ad9120, (0x20d6 bytes) 2024-05-08T08:25:09.352 INFO:teuthology.orchestra.run.smithi031.stdout:148 write 0x337eddc thru 0x33802f5 (0x151a bytes) 2024-05-08T08:25:09.362 INFO:teuthology.orchestra.run.smithi031.stdout:149 write 0x42d1929 thru 0x42dba4f (0xa127 bytes) 2024-05-08T08:25:09.369 INFO:teuthology.orchestra.run.smithi031.stdout:151 read 0x31a5671 thru 0x31a6044 (0x9d4 bytes) 2024-05-08T08:25:09.369 INFO:teuthology.orchestra.run.smithi031.stdout:152 writesame 0x9794f52 thru 0x97a0811 data_size 0x20(0xb8c0 bytes) 2024-05-08T08:25:09.380 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.380 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.385 INFO:teuthology.orchestra.run.smithi031.stdout:155 trunc from 0x9ef863a to 0x531dece 2024-05-08T08:25:09.396 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:09.403+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:09.399 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:09.399 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:09.405 INFO:teuthology.orchestra.run.smithi031.stdout:157 punch from 0x2403235 to 0x24102b0, (0xd07b bytes) 2024-05-08T08:25:09.405 INFO:teuthology.orchestra.run.smithi031.stdout:159 read 0x2ddbfb thru 0x2eb6c8 (0xdace bytes) 2024-05-08T08:25:09.405 INFO:teuthology.orchestra.run.smithi031.stdout:160 read 0x376d87d thru 0x3770f9e (0x3722 bytes) 2024-05-08T08:25:09.408 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:09.414 INFO:teuthology.orchestra.run.smithi031.stdout:161 write 0x78b3861 thru 0x78b5053 (0x17f3 bytes) 2024-05-08T08:25:09.422 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.434 INFO:teuthology.orchestra.run.smithi031.stdout:163 writesame 0xcc9d994 thru 0xcca0697 data_size 0x43(0x2d04 bytes) 2024-05-08T08:25:09.434 INFO:teuthology.orchestra.run.smithi031.stdout:164 writesame 0x584d189 thru 0x5856751 data_size 0x1(0x95c9 bytes) 2024-05-08T08:25:09.446 INFO:teuthology.orchestra.run.smithi031.stdout:165 punch from 0xb270f29 to 0xb27ee60, (0xdf37 bytes) 2024-05-08T08:25:09.446 INFO:teuthology.orchestra.run.smithi031.stdout:166 punch from 0x137c606 to 0x1383c82, (0x767c bytes) 2024-05-08T08:25:09.447 INFO:teuthology.orchestra.run.smithi031.stdout:167 punch from 0x4ee96bf to 0x4eeb478, (0x1db9 bytes) 2024-05-08T08:25:09.447 INFO:teuthology.orchestra.run.smithi031.stdout:169 read 0x9df2550 thru 0x9dfb822 (0x92d3 bytes) 2024-05-08T08:25:09.447 INFO:teuthology.orchestra.run.smithi031.stdout:172 write 0xb5439f1 thru 0xb54f8ef (0xbeff bytes) 2024-05-08T08:25:09.456 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.464 INFO:teuthology.orchestra.run.smithi031.stdout:175 write 0xe7eeaf1 thru 0xe7f7efe (0x940e bytes) 2024-05-08T08:25:09.474 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.474 INFO:teuthology.orchestra.run.smithi031.stdout:181 write 0x2b5fdde thru 0x2b64ab7 (0x4cda bytes) 2024-05-08T08:25:09.482 INFO:teuthology.orchestra.run.smithi031.stdout:182 write 0x4b4aaa5 thru 0x4b53eed (0x9449 bytes) 2024-05-08T08:25:09.493 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.497 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:09.505+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:09.504 INFO:teuthology.orchestra.run.smithi031.stdout:184 trunc from 0xe7f7eff to 0x4e6b5ec 2024-05-08T08:25:09.525 INFO:teuthology.orchestra.run.smithi031.stdout:185 punch from 0x3ba6d8c to 0x3bab63a, (0x48ae bytes) 2024-05-08T08:25:09.527 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:09.527 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:09.533 INFO:teuthology.orchestra.run.smithi031.stdout:187 write 0x6db675f thru 0x6dc3ec1 (0xd763 bytes) 2024-05-08T08:25:09.537 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:09.559 INFO:teuthology.orchestra.run.smithi031.stdout:188 write 0xc6123b5 thru 0xc621292 (0xeede bytes) 2024-05-08T08:25:09.571 INFO:teuthology.orchestra.run.smithi031.stdout:190 punch from 0xba4582f to 0xba4942c, (0x3bfd bytes) 2024-05-08T08:25:09.572 INFO:teuthology.orchestra.run.smithi031.stdout:191 write 0x92337e2 thru 0x923b1c4 (0x79e3 bytes) 2024-05-08T08:25:09.591 INFO:teuthology.orchestra.run.smithi031.stdout:193 trunc from 0xc621293 to 0x51326b8 2024-05-08T08:25:09.618 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:25:09.621 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:09.629+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:09.631 INFO:teuthology.orchestra.run.smithi031.stdout:194 write 0xb91bb4b thru 0xb9228e8 (0x6d9e bytes) 2024-05-08T08:25:09.646 INFO:teuthology.orchestra.run.smithi031.stdout:195 punch from 0x25b2faa to 0x25b7b73, (0x4bc9 bytes) 2024-05-08T08:25:09.646 INFO:teuthology.orchestra.run.smithi031.stdout:198 punch from 0x98a5ae0 to 0x98aafea, (0x550a bytes) 2024-05-08T08:25:09.646 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.647 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.647 INFO:teuthology.orchestra.run.smithi031.stdout:201 read 0x8ad58a8 thru 0x8ade1b0 (0x8909 bytes) 2024-05-08T08:25:09.648 INFO:teuthology.orchestra.run.smithi031.stdout:202 trunc from 0xb9228e9 to 0x9c774ce 2024-05-08T08:25:09.665 INFO:teuthology.orchestra.run.smithi031.stdout:203 write 0x14b2161 thru 0x14b7bfe (0x5a9e bytes) 2024-05-08T08:25:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.680 INFO:teuthology.orchestra.run.smithi031.stdout:206 punch from 0x8b1be0d to 0x8b1f9da, (0x3bcd bytes) 2024-05-08T08:25:09.687 INFO:teuthology.orchestra.run.smithi031.stdout:210 trunc from 0x9c774ce to 0x321fe2e 2024-05-08T08:25:09.694 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:09.694 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:09.699 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:09.706+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:09.701 INFO:teuthology.orchestra.run.smithi031.stdout:212 read 0x841406 thru 0x8463e3 (0x4fde bytes) 2024-05-08T08:25:09.704 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:09.716 INFO:teuthology.orchestra.run.smithi031.stdout:213 write 0xac09362 thru 0xac0e908 (0x55a7 bytes) 2024-05-08T08:25:09.724 INFO:teuthology.orchestra.run.smithi031.stdout:215 write 0x88f3e2d thru 0x88fd872 (0x9a46 bytes) 2024-05-08T08:25:09.734 INFO:teuthology.orchestra.run.smithi031.stdout:216 write 0xaa01acc thru 0xaa0b280 (0x97b5 bytes) 2024-05-08T08:25:09.745 INFO:teuthology.orchestra.run.smithi031.stdout:217 write 0x3c69118 thru 0x3c728a0 (0x9789 bytes) 2024-05-08T08:25:09.761 INFO:teuthology.orchestra.run.smithi031.stdout:218 trunc from 0xac0e909 to 0x742f9dc 2024-05-08T08:25:09.786 INFO:teuthology.orchestra.run.smithi031.stdout:219 writesame 0xa510174 thru 0xa513333 data_size 0x8(0x31c0 bytes) 2024-05-08T08:25:09.790 INFO:teuthology.orchestra.run.smithi031.stdout:220 trunc from 0xa513334 to 0x6b907f7 2024-05-08T08:25:09.797 INFO:teuthology.orchestra.run.smithi031.stdout:221 read 0x1c8dafc thru 0x1c9add0 (0xd2d5 bytes) 2024-05-08T08:25:09.797 INFO:teuthology.orchestra.run.smithi031.stdout:222 read 0x747f3 thru 0x7afae (0x67bc bytes) 2024-05-08T08:25:09.797 INFO:teuthology.orchestra.run.smithi031.stdout:223 read 0x60e3039 thru 0x60f1202 (0xe1ca bytes) 2024-05-08T08:25:09.797 INFO:teuthology.orchestra.run.smithi031.stdout:225 read 0x671c81 thru 0x676ac5 (0x4e45 bytes) 2024-05-08T08:25:09.797 INFO:teuthology.orchestra.run.smithi031.stdout:228 trunc from 0x6b907f7 to 0x649aea8 2024-05-08T08:25:09.799 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:09.807+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:09.803 INFO:teuthology.orchestra.run.smithi031.stdout:229 read 0x3b4908 thru 0x3be913 (0xa00c bytes) 2024-05-08T08:25:09.803 INFO:teuthology.orchestra.run.smithi031.stdout:230 read 0x61014b1 thru 0x6108b82 (0x76d2 bytes) 2024-05-08T08:25:09.818 INFO:teuthology.orchestra.run.smithi031.stdout:231 write 0xd43cf76 thru 0xd44adba (0xde45 bytes) 2024-05-08T08:25:09.828 INFO:teuthology.orchestra.run.smithi031.stdout:232 punch from 0x1f9cd01 to 0x1fa627e, (0x957d bytes) 2024-05-08T08:25:09.828 INFO:teuthology.orchestra.run.smithi031.stdout:233 punch from 0x89f0194 to 0x89f1167, (0xfd3 bytes) 2024-05-08T08:25:09.834 INFO:teuthology.orchestra.run.smithi031.stdout:234 write 0xe2ed6a5 thru 0xe2eddc5 (0x721 bytes) 2024-05-08T08:25:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:235 punch from 0x1a52bf to 0x1ab15e, (0x5e9f bytes) 2024-05-08T08:25:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:236 read 0x63c5595 thru 0x63ceedd (0x9949 bytes) 2024-05-08T08:25:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.839 INFO:teuthology.orchestra.run.smithi031.stdout:239 write 0xc9acac6 thru 0xc9b9d2c (0xd267 bytes) 2024-05-08T08:25:09.845 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:09.846 INFO:teuthology.orchestra.run.smithi031.stdout:240 punch from 0x2e9fa08 to 0x2ea77e9, (0x7de1 bytes) 2024-05-08T08:25:09.846 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.846 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.855 INFO:teuthology.orchestra.run.smithi031.stdout:243 trunc from 0xe2eddc6 to 0x5e28fcf 2024-05-08T08:25:09.866 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:09.866 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:09.874 INFO:teuthology.orchestra.run.smithi031.stdout:244 trunc from 0x5e28fcf to 0x5fbf93e 2024-05-08T08:25:09.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:09.879 INFO:teuthology.orchestra.run.smithi031.stdout:245 read 0x5be3a04 thru 0x5bf3163 (0xf760 bytes) 2024-05-08T08:25:09.884 INFO:teuthology.orchestra.run.smithi031.stdout:246 trunc from 0x5fbf93e to 0x26347a8 2024-05-08T08:25:09.900 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:09.907+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:09.907 INFO:teuthology.orchestra.run.smithi031.stdout:249 read 0x8d0647 thru 0x8d535d (0x4d17 bytes) 2024-05-08T08:25:09.907 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.910 INFO:teuthology.orchestra.run.smithi031.stdout:251 trunc from 0x26347a8 to 0x565d798 2024-05-08T08:25:09.917 INFO:teuthology.orchestra.run.smithi031.stdout:252 trunc from 0x565d798 to 0x42b998c 2024-05-08T08:25:09.932 INFO:teuthology.orchestra.run.smithi031.stdout:254 write 0x771f528 thru 0x77284e7 (0x8fc0 bytes) 2024-05-08T08:25:09.947 INFO:teuthology.orchestra.run.smithi031.stdout:255 read 0x410a8d5 thru 0x411a76c (0xfe98 bytes) 2024-05-08T08:25:09.947 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.947 INFO:teuthology.orchestra.run.smithi031.stdout:257 read 0x93d6f thru 0x9e28c (0xa51e bytes) 2024-05-08T08:25:09.948 INFO:teuthology.orchestra.run.smithi031.stdout:258 punch from 0x871794 to 0x87c553, (0xadbf bytes) 2024-05-08T08:25:09.948 INFO:teuthology.orchestra.run.smithi031.stdout:259 read 0x71a51be thru 0x71aa6ce (0x5511 bytes) 2024-05-08T08:25:09.948 INFO:teuthology.orchestra.run.smithi031.stdout:260 write 0x60dfe2c thru 0x60e7edc (0x80b1 bytes) 2024-05-08T08:25:09.960 INFO:teuthology.orchestra.run.smithi031.stdout:261 punch from 0x657f1b1 to 0x6583838, (0x4687 bytes) 2024-05-08T08:25:09.969 INFO:teuthology.orchestra.run.smithi031.stdout:262 write 0xa1240a0 thru 0xa12963c (0x559d bytes) 2024-05-08T08:25:09.983 INFO:teuthology.orchestra.run.smithi031.stdout:263 read 0x316bda2 thru 0x31797bd (0xda1c bytes) 2024-05-08T08:25:09.983 INFO:teuthology.orchestra.run.smithi031.stdout:264 write 0x6179d19 thru 0x61830db (0x93c3 bytes) 2024-05-08T08:25:09.996 INFO:teuthology.orchestra.run.smithi031.stdout:265 punch from 0x8c0f73d to 0x8c18391, (0x8c54 bytes) 2024-05-08T08:25:09.996 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:09.996 INFO:teuthology.orchestra.run.smithi031.stdout:269 read 0x6ac239c thru 0x6acb2e7 (0x8f4c bytes) 2024-05-08T08:25:09.996 INFO:teuthology.orchestra.run.smithi031.stdout:271 write 0x8a32264 thru 0x8a41e3d (0xfbda bytes) 2024-05-08T08:25:09.998 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:09.998 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:10.001 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:10.008+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:10.004 INFO:teuthology.orchestra.run.smithi031.stdout:272 writesame 0x1a3f843 thru 0x1a4f759 data_size 0x1(0xff17 bytes) 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1572 3.10 e23/18 3:0f63064c:::rbd_data.119116cc7dd0.000000000000006a:head v 23'37, mlcod=23'35)", 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.023330+0000", 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00043613700000000002, 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00045958300000000001, 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023330+0000", 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023330+0000", 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023331+0000", 2024-05-08T08:25:10.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1399999999999996e-07 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023624+0000", 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00029319300000000002 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023637+0000", 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3237e-05 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023646+0000", 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5830000000000007e-06 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023702+0000", 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6427e-05 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.023728+0000", 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5880999999999998e-05 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:10.020 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-05-08T08:25:10.020 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:10.020 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:10.020 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:273 read 0x43944f7 thru 0x4396b97 (0x26a1 bytes) 2024-05-08T08:25:10.021 INFO:teuthology.orchestra.run.smithi031.stdout:274 read 0x132aa18 thru 0x132e2e9 (0x38d2 bytes) 2024-05-08T08:25:10.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:10.031 INFO:teuthology.orchestra.run.smithi031.stdout:276 trunc from 0xa12963d to 0x84e204 2024-05-08T08:25:10.044 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:10.045 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:10.067 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:10.067 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:10.086 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:10.086 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:10.094 INFO:teuthology.orchestra.run.smithi031.stdout:278 write 0x97dec18 thru 0x97ee2e3 (0xf6cc bytes) 2024-05-08T08:25:10.102 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:10.110+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:10.103 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.103 INFO:teuthology.orchestra.run.smithi031.stdout:280 writesame 0x3b597b9 thru 0x3b5989b data_size 0x1(0xe3 bytes) 2024-05-08T08:25:10.107 INFO:teuthology.orchestra.run.smithi031.stdout:281 punch from 0x24efc2a to 0x24faf4c, (0xb322 bytes) 2024-05-08T08:25:10.107 INFO:teuthology.orchestra.run.smithi031.stdout:282 writesame 0x6259d13 thru 0x6267d6a data_size 0xc(0xe058 bytes) 2024-05-08T08:25:10.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:10.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:283 trunc from 0x97ee2e4 to 0x5fd15d2 2024-05-08T08:25:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:10.129 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:284 read 0x2d0328e thru 0x2d0a11c (0x6e8f bytes) 2024-05-08T08:25:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:286 read 0x3c41052 thru 0x3c48a06 (0x79b5 bytes) 2024-05-08T08:25:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:287 read 0x106481b thru 0x1068a72 (0x4258 bytes) 2024-05-08T08:25:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:10.151 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:288 write 0xb8341c5 thru 0xb838e96 (0x4cd2 bytes) 2024-05-08T08:25:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:10.161 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:291 write 0x50404d thru 0x50f54d (0xb501 bytes) 2024-05-08T08:25:10.173 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:292 read 0x9413806 thru 0x9414d18 (0x1513 bytes) 2024-05-08T08:25:10.181 INFO:teuthology.orchestra.run.smithi031.stdout:294 write 0x651f1f thru 0x659810 (0x78f2 bytes) 2024-05-08T08:25:10.189 INFO:teuthology.orchestra.run.smithi031.stdout:296 write 0x5dd9439 thru 0x5ddd684 (0x424c bytes) 2024-05-08T08:25:10.197 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.203 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:10.210+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:10.204 INFO:teuthology.orchestra.run.smithi031.stdout:298 write 0xd48f9c6 thru 0xd49cd40 (0xd37b bytes) 2024-05-08T08:25:10.212 INFO:teuthology.orchestra.run.smithi031.stdout:300 write 0x2cb2483 thru 0x2cb32fb (0xe79 bytes) 2024-05-08T08:25:10.217 INFO:teuthology.orchestra.run.smithi031.stdout:301 write 0x2182558 thru 0x2184c52 (0x26fb bytes) 2024-05-08T08:25:10.228 INFO:teuthology.orchestra.run.smithi031.stdout:302 read 0xc958ac8 thru 0xc9619bf (0x8ef8 bytes) 2024-05-08T08:25:10.228 INFO:teuthology.orchestra.run.smithi031.stdout:303 writesame 0x51c3a66 thru 0x51cef1b data_size 0x1(0xb4b6 bytes) 2024-05-08T08:25:10.239 INFO:teuthology.orchestra.run.smithi031.stdout:304 write 0x7936826 thru 0x7943fd3 (0xd7ae bytes) 2024-05-08T08:25:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:305 read 0x1cf3a26 thru 0x1cfe95b (0xaf36 bytes) 2024-05-08T08:25:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:307 write 0x95abf56 thru 0x95b476a (0x8815 bytes) 2024-05-08T08:25:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:310 trunc from 0xd49cd41 to 0xa506203 2024-05-08T08:25:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:311 trunc from 0xa506203 to 0xe53d2b6 2024-05-08T08:25:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:312 trunc from 0xe53d2b6 to 0x99ed374 2024-05-08T08:25:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:313 read 0x15a1ede thru 0x15aab11 (0x8c34 bytes) 2024-05-08T08:25:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:315 read 0x82c313b thru 0x82c9ec9 (0x6d8f bytes) 2024-05-08T08:25:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:317 read 0x2956987 thru 0x2963457 (0xcad1 bytes) 2024-05-08T08:25:10.304 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:10.311+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:11 3.1f 3:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:06.625816+0000", 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6923833319999999, 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0076387319999999996, 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.312 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 11 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625816+0000", 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625816+0000", 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625818+0000", 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.66e-06 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625827+0000", 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8550000000000008e-06 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625847+0000", 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9884000000000001e-05 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625857+0000", 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0288999999999999e-05 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625928+0000", 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0804999999999997e-05 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.625933+0000", 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0810000000000001e-06 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.627906+0000", 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019726890000000001 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.628058+0000", 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015223799999999999 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.628285+0000", 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022732899999999999 2024-05-08T08:25:10.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.631748+0000", 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034626180000000002 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.633381+0000", 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001633219 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.633422+0000", 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0694e-05 2024-05-08T08:25:10.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:06.633455+0000", 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3371000000000003e-05 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1565 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.012642+0000", 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.30555796299999999, 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078488350000000002, 2024-05-08T08:25:10.318 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1565 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012642+0000", 2024-05-08T08:25:10.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012642+0000", 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012642+0000", 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012645+0000", 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5790000000000002e-06 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012650+0000", 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4489999999999998e-06 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012654+0000", 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4359999999999998e-06 2024-05-08T08:25:10.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.014420+0000", 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017663679999999999 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.014465+0000", 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5087999999999999e-05 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.014653+0000", 2024-05-08T08:25:10.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018756 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.016399+0000", 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001746505 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020408+0000", 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040092610000000001 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020451+0000", 2024-05-08T08:25:10.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2512000000000002e-05 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020491+0000", 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9684999999999997e-05 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1566 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.012659+0000", 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.30554120800000001, 2024-05-08T08:25:10.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079209510000000007, 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1566 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012659+0000", 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012659+0000", 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012659+0000", 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.35e-07 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012660+0000", 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6479999999999999e-06 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012664+0000", 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4970000000000001e-06 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012666+0000", 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5229999999999998e-06 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.015321+0000", 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002654642 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.015366+0000", 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4790999999999999e-05 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.015552+0000", 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000186069 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.016926+0000", 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013741949999999999 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020501+0000", 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035748030000000001 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020544+0000", 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3458000000000003e-05 2024-05-08T08:25:10.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020579+0000", 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5089999999999998e-05 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1567 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.012670+0000", 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.30553005100000002, 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079766490000000006, 2024-05-08T08:25:10.330 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1567 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012670+0000", 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012670+0000", 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012670+0000", 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012672+0000", 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8419999999999999e-06 2024-05-08T08:25:10.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012676+0000", 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4070000000000003e-06 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.012679+0000", 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.853e-06 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.015839+0000", 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031601649999999999 2024-05-08T08:25:10.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.015891+0000", 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2244999999999999e-05 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.016080+0000", 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018825600000000001 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.017823+0000", 2024-05-08T08:25:10.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001743524 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020585+0000", 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00276165 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020621+0000", 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5725000000000001e-05 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.020646+0000", 2024-05-08T08:25:10.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5775e-05 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1635 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.068446+0000", 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.24975349799999999, 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086399719999999992, 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.336 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1635 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.068446+0000", 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.068446+0000", 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.068447+0000", 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4200000000000005e-07 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.068452+0000", 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4199999999999998e-06 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.068463+0000", 2024-05-08T08:25:10.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0387999999999999e-05 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.068470+0000", 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2699999999999999e-06 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.071655+0000", 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031850899999999998 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.071742+0000", 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6700000000000007e-05 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.072006+0000", 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00026379600000000001 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.076596+0000", 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045906690000000003 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.076997+0000", 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040023899999999999 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.077048+0000", 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1523000000000001e-05 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.077086+0000", 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8034999999999998e-05 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.341 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1664 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.102844+0000", 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.21535616299999999, 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0072865819999999998, 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.342 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1664 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102844+0000", 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102844+0000", 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102844+0000", 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102846+0000", 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1720000000000001e-06 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102849+0000", 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1719999999999999e-06 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102852+0000", 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105337+0000", 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024852950000000002 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105382+0000", 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4928000000000001e-05 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105705+0000", 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00032315000000000002 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.107429+0000", 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017241800000000001 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.110058+0000", 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026293169999999999 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.110089+0000", 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0226000000000002e-05 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.110130+0000", 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1439000000000001e-05 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1665 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.102855+0000", 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.21534497599999999, 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074123549999999998, 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1665 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102855+0000", 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102855+0000", 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102855+0000", 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102859+0000", 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9140000000000001e-06 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102862+0000", 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1379999999999999e-06 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.102864+0000", 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4719999999999998e-06 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.106086+0000", 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003221823 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.106243+0000", 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000156816 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.106655+0000", 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00041145500000000002 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.109014+0000", 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023591929999999999 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.110181+0000", 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001167529 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.110232+0000", 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0569000000000003e-05 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.110267+0000", 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5237999999999997e-05 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1679 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.116238+0000", 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.201961469, 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075123150000000003, 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.354 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1679 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116238+0000", 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116238+0000", 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116239+0000", 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7000000000000004e-07 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116243+0000", 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9530000000000001e-06 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116252+0000", 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0010000000000007e-06 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116259+0000", 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4800000000000004e-06 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.119132+0000", 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028723770000000002 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.119215+0000", 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3466000000000004e-05 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.119604+0000", 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038882599999999999 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.122347+0000", 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027428140000000001 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.123674+0000", 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001327504 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.123708+0000", 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3578999999999998e-05 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.123751+0000", 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2645000000000003e-05 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1680 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.116295+0000", 2024-05-08T08:25:10.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.201904529, 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010741716, 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1680 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116295+0000", 2024-05-08T08:25:10.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116295+0000", 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116296+0000", 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6700000000000003e-07 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116300+0000", 2024-05-08T08:25:10.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8859999999999997e-06 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116308+0000", 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9759999999999995e-06 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116315+0000", 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9530000000000002e-06 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.120214+0000", 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038989749999999998 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.120299+0000", 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5319000000000001e-05 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.120730+0000", 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00043119099999999999 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.122394+0000", 2024-05-08T08:25:10.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001664303 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.126919+0000", 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045249449999999998 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.126991+0000", 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1731999999999997e-05 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127037+0000", 2024-05-08T08:25:10.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5868999999999997e-05 2024-05-08T08:25:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:319 write 0xdd24016 thru 0xdd2e60f (0xa5fa bytes) 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1681 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.116379+0000", 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.201820689, 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010794981, 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.367 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1681 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116379+0000", 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116379+0000", 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116380+0000", 2024-05-08T08:25:10.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4000000000000001e-07 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116384+0000", 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0210000000000001e-06 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116393+0000", 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1509999999999993e-06 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.116400+0000", 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.486e-06 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121423+0000", 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050230709999999996 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121509+0000", 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5395999999999994e-05 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121895+0000", 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00038617099999999999 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.122717+0000", 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00082173200000000004 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127093+0000", 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043761370000000004 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127141+0000", 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7701000000000003e-05 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127174+0000", 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3475000000000001e-05 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1724 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.372 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.165530+0000", 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.152669316, 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083633470000000001, 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1724 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165530+0000", 2024-05-08T08:25:10.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165530+0000", 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165531+0000", 2024-05-08T08:25:10.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165534+0000", 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4929999999999998e-06 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165541+0000", 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8569999999999997e-06 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165546+0000", 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1930000000000001e-06 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.169634+0000", 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040879280000000002 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.169700+0000", 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5809000000000006e-05 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.169938+0000", 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023758399999999999 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171443+0000", 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015054420000000001 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173785+0000", 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002341536 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173841+0000", 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6758000000000001e-05 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173894+0000", 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2280000000000001e-05 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1725 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.165556+0000", 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.15264420100000001, 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086766199999999995, 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1725 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165556+0000", 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165556+0000", 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165556+0000", 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-05-08T08:25:10.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165560+0000", 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7419999999999999e-06 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165571+0000", 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1187e-05 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.165576+0000", 2024-05-08T08:25:10.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5720000000000002e-06 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.170569+0000", 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049922020000000003 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.170661+0000", 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2522000000000006e-05 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.170869+0000", 2024-05-08T08:25:10.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020751100000000001 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172010+0000", 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011409860000000001 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174157+0000", 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021477129999999999 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174204+0000", 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6428999999999999e-05 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174232+0000", 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8336000000000001e-05 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1736 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.177430+0000", 2024-05-08T08:25:10.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14076942000000001, 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0082455489999999996, 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1736 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177430+0000", 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177430+0000", 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177431+0000", 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4200000000000004e-07 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177437+0000", 2024-05-08T08:25:10.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4770000000000002e-06 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177447+0000", 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7969999999999995e-06 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177451+0000", 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179677+0000", 2024-05-08T08:25:10.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022255209999999998 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179777+0000", 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010027299999999999 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.180179+0000", 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040206600000000002 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.180847+0000", 2024-05-08T08:25:10.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00066748199999999995 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.185602+0000", 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047554140000000003 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.185649+0000", 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6558999999999999e-05 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.185676+0000", 2024-05-08T08:25:10.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7194000000000001e-05 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1737 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.177704+0000", 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.14049561399999999, 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080955639999999995, 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1737 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177704+0000", 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177704+0000", 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177705+0000", 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177708+0000", 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.45e-06 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177715+0000", 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.177720+0000", 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2799999999999997e-06 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.180878+0000", 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031577279999999998 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.180986+0000", 2024-05-08T08:25:10.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108497 2024-05-08T08:25:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.181346+0000", 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000359551 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.182143+0000", 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00079792999999999997 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.185733+0000", 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003589933 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.185775+0000", 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1894999999999998e-05 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.185800+0000", 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4360999999999999e-05 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1811 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.236718+0000", 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.081481782000000003, 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073231650000000004, 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.397 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1811 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236718+0000", 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236718+0000", 2024-05-08T08:25:10.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236718+0000", 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236721+0000", 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8559999999999998e-06 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236727+0000", 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0530000000000003e-06 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236733+0000", 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000001e-06 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.239196+0000", 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024625570000000002 2024-05-08T08:25:10.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.239247+0000", 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1987000000000002e-05 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.239448+0000", 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020054299999999999 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.241403+0000", 2024-05-08T08:25:10.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0019548009999999999 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.243937+0000", 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025341299999999999 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.243998+0000", 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1439999999999995e-05 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.244041+0000", 2024-05-08T08:25:10.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2704000000000003e-05 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1812 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.236743+0000", 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.081456516000000007, 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073992520000000003, 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1812 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236743+0000", 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236743+0000", 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236744+0000", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236748+0000", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8890000000000004e-06 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236755+0000", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0339999999999999e-06 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236760+0000", 2024-05-08T08:25:10.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9590000000000003e-06 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.239910+0000", 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031505859999999999 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.239972+0000", 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1754999999999994e-05 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.240176+0000", 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020381800000000001 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.241437+0000", 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012609609999999999 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.244050+0000", 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026136699999999998 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.244103+0000", 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3090999999999997e-05 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.244142+0000", 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9078000000000001e-05 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1822 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.248441+0000", 2024-05-08T08:25:10.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.069758300999999995, 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078785260000000003, 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1822 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248441+0000", 2024-05-08T08:25:10.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248441+0000", 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248442+0000", 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248444+0000", 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1110000000000002e-06 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248449+0000", 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3990000000000003e-06 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248452+0000", 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.869e-06 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251371+0000", 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002918449 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251433+0000", 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2147e-05 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251670+0000", 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00023721500000000001 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.253339+0000", 2024-05-08T08:25:10.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001668987 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256264+0000", 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029249900000000001 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256296+0000", 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1820999999999999e-05 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256320+0000", 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4168999999999998e-05 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1823 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.248456+0000", 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.069743880999999994, 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0079118420000000005, 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1823 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248456+0000", 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248456+0000", 2024-05-08T08:25:10.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248456+0000", 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248458+0000", 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7349999999999999e-06 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248461+0000", 2024-05-08T08:25:10.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.304e-06 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248465+0000", 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9319999999999998e-06 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.252221+0000", 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037561159999999999 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.252313+0000", 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1472999999999996e-05 2024-05-08T08:25:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.252740+0000", 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000427308 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.254172+0000", 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00143213 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256326+0000", 2024-05-08T08:25:10.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021540550000000002 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256350+0000", 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3836999999999999e-05 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256368+0000", 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7728e-05 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1824 3.1b 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.248471+0000", 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.069728453999999995, 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010692479, 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1824 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248471+0000", 2024-05-08T08:25:10.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248471+0000", 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248472+0000", 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248473+0000", 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8220000000000001e-06 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248477+0000", 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6119999999999999e-06 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.248482+0000", 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8910000000000003e-06 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.253368+0000", 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048858570000000004 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.253466+0000", 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8412999999999994e-05 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.253736+0000", 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000269611 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.254695+0000", 2024-05-08T08:25:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00095893900000000002 2024-05-08T08:25:10.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259067+0000", 2024-05-08T08:25:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043726420000000004 2024-05-08T08:25:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259119+0000", 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1252999999999999e-05 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259164+0000", 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5113e-05 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1826 3.3 e23/18 3:c3deaffe:::rbd_data.119116cc7dd0.00000000000001e1:head v 23'48, mlcod=23'46)", 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.251763+0000", 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.066436861, 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074315589999999999, 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251763+0000", 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251763+0000", 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251764+0000", 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8700000000000005e-07 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251777+0000", 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3932e-05 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251789+0000", 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1697e-05 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251797+0000", 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.52e-06 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.253271+0000", 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014739519999999999 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.253289+0000", 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8638999999999999e-05 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259168+0000", 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0058785189999999996 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259194+0000", 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6613000000000001e-05 2024-05-08T08:25:10.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.431 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:10.431 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:321 punch from 0x85b7c36 to 0x85bd56b, (0x5935 bytes) 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:323 write 0xe757a4f thru 0xe7621a6 (0xa758 bytes) 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:324 read 0x4037f64 thru 0x4041ea5 (0x9f42 bytes) 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:328 read 0x4e6a2fe thru 0x4e72861 (0x8564 bytes) 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:330 writesame 0xec15a8 thru 0xece21f data_size 0x24(0xcc78 bytes) 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:331 punch from 0x42a6f6c to 0x42a8612, (0x16a6 bytes) 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:333 read 0x23dbc8d thru 0x23de01d (0x2391 bytes) 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:335 trunc from 0xe7621a7 to 0xb0dcafd 2024-05-08T08:25:10.434 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.435 INFO:teuthology.orchestra.run.smithi031.stdout:337 writesame 0xd180690 thru 0xd186ac0 data_size 0x1(0x6431 bytes) 2024-05-08T08:25:10.435 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.435 INFO:teuthology.orchestra.run.smithi031.stdout:339 read 0x4c61a27 thru 0x4c6a770 (0x8d4a bytes) 2024-05-08T08:25:10.435 INFO:teuthology.orchestra.run.smithi031.stdout:340 trunc from 0xd186ac1 to 0x8f41d8 2024-05-08T08:25:10.435 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:10.441+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:10.442 INFO:teuthology.orchestra.run.smithi031.stdout:341 write 0x9f4b29f thru 0x9f4c570 (0x12d2 bytes) 2024-05-08T08:25:10.446 INFO:teuthology.orchestra.run.smithi031.stdout:342 read 0x6413fd8 thru 0x6417008 (0x3031 bytes) 2024-05-08T08:25:10.446 INFO:teuthology.orchestra.run.smithi031.stdout:343 writesame 0x8ea0a9 thru 0x8f6e9e data_size 0x29(0xcdf6 bytes) 2024-05-08T08:25:10.450 INFO:teuthology.orchestra.run.smithi031.stdout:344 trunc from 0x9f4c571 to 0xa72edbd 2024-05-08T08:25:10.466 INFO:teuthology.orchestra.run.smithi031.stdout:345 write 0xe6bd100 thru 0xe6cc335 (0xf236 bytes) 2024-05-08T08:25:10.474 INFO:teuthology.orchestra.run.smithi031.stdout:346 writesame 0x65aee76 thru 0x65b1269 data_size 0x3b(0x23f4 bytes) 2024-05-08T08:25:10.483 INFO:teuthology.orchestra.run.smithi031.stdout:349 writesame 0xc99c72b thru 0xc9a4178 data_size 0xa(0x7a4e bytes) 2024-05-08T08:25:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:351 read 0x5b0f34c thru 0x5b1ccfc (0xd9b1 bytes) 2024-05-08T08:25:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:352 writesame 0x29a492f thru 0x29af846 data_size 0x1a(0xaf18 bytes) 2024-05-08T08:25:10.504 INFO:teuthology.orchestra.run.smithi031.stdout:353 write 0x55d8769 thru 0x55e028a (0x7b22 bytes) 2024-05-08T08:25:10.530 INFO:teuthology.orchestra.run.smithi031.stdout:356 trunc from 0xe6cc336 to 0x4f437d3 2024-05-08T08:25:10.535 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:10.542+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:10.553 INFO:teuthology.orchestra.run.smithi031.stdout:357 punch from 0x2a892c0 to 0x2a95dc2, (0xcb02 bytes) 2024-05-08T08:25:10.554 INFO:teuthology.orchestra.run.smithi031.stdout:359 write 0x311550e thru 0x31229fe (0xd4f1 bytes) 2024-05-08T08:25:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:10.567 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:10.567 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:10.571 INFO:teuthology.orchestra.run.smithi031.stdout:360 write 0x81079e0 thru 0x8110221 (0x8842 bytes) 2024-05-08T08:25:10.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:10.587 INFO:teuthology.orchestra.run.smithi031.stdout:361 write 0x335b2f6 thru 0x336999e (0xe6a9 bytes) 2024-05-08T08:25:10.595 INFO:teuthology.orchestra.run.smithi031.stdout:362 read 0x2120865 thru 0x2126df5 (0x6591 bytes) 2024-05-08T08:25:10.595 INFO:teuthology.orchestra.run.smithi031.stdout:364 punch from 0x76f92ba to 0x7700035, (0x6d7b bytes) 2024-05-08T08:25:10.595 INFO:teuthology.orchestra.run.smithi031.stdout:365 read 0x2b08038 thru 0x2b0e3d1 (0x639a bytes) 2024-05-08T08:25:10.607 INFO:teuthology.orchestra.run.smithi031.stdout:367 write 0xe2d0fdf thru 0xe2da07b (0x909d bytes) 2024-05-08T08:25:10.620 INFO:teuthology.orchestra.run.smithi031.stdout:370 punch from 0xa5e4f53 to 0xa5f19c2, (0xca6f bytes) 2024-05-08T08:25:10.620 INFO:teuthology.orchestra.run.smithi031.stdout:371 write 0x799edb0 thru 0x79ac482 (0xd6d3 bytes) 2024-05-08T08:25:10.634 INFO:teuthology.orchestra.run.smithi031.stdout:372 read 0x7e7dad0 thru 0x7e8bd8f (0xe2c0 bytes) 2024-05-08T08:25:10.634 INFO:teuthology.orchestra.run.smithi031.stdout:373 read 0x5fcbcfc thru 0x5fcc23c (0x541 bytes) 2024-05-08T08:25:10.634 INFO:teuthology.orchestra.run.smithi031.stdout:374 read 0x7615902 thru 0x7625350 (0xfa4f bytes) 2024-05-08T08:25:10.634 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.634 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:10.634 INFO:teuthology.orchestra.run.smithi031.stdout:378 read 0x4bbe546 thru 0x4bc280e (0x42c9 bytes) 2024-05-08T08:25:10.634 INFO:teuthology.orchestra.run.smithi031.stdout:379 clone 2 order 25 su 262144 sc 4 2024-05-08T08:25:10.635 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:10.643+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:10.704 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:10.704 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:10.704 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:10.713 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:10.754 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:10.761+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:10.837 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:10.845+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:10.863 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:1 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.248752966, 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075841290000000002, 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621510+0000", 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9610000000000002e-06 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621517+0000", 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0949999999999998e-06 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621526+0000", 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-05-08T08:25:10.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621572+0000", 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5599000000000002e-05 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621600+0000", 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8399e-05 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.629024+0000", 2024-05-08T08:25:10.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074240119999999998 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.629088+0000", 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3455999999999997e-05 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:83 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-05-08T08:25:10.868 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.148568157, 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052916229999999996, 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721689+0000", 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721697+0000", 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721706+0000", 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0000000000000002e-06 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721713+0000", 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2380000000000002e-06 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721822+0000", 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010911 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721845+0000", 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3833999999999999e-05 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726930+0000", 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050846830000000004 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726980+0000", 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9957999999999998e-05 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:84 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.148100722000001, 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048602560000000003, 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722157+0000", 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7899999999999998e-07 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722191+0000", 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4762999999999999e-05 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722201+0000", 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2380000000000003e-06 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722209+0000", 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8019999999999998e-06 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722251+0000", 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1128999999999999e-05 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722268+0000", 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7371e-05 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726987+0000", 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047187699999999997 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727016+0000", 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9504000000000001e-05 2024-05-08T08:25:10.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:85 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'55, mlcod=16'50)", 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.147672739000001, 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044628130000000004, 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722584+0000", 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7400000000000003e-07 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722593+0000", 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8070000000000006e-06 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722602+0000", 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0920000000000004e-06 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722609+0000", 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1650000000000003e-06 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722733+0000", 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012490099999999999 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722756+0000", 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2819e-05 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727018+0000", 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042616590000000001 2024-05-08T08:25:10.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727047+0000", 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8796e-05 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:104 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.919836+0000", 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.950420357, 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042493720000000004, 2024-05-08T08:25:10.881 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919836+0000", 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919836+0000", 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919838+0000", 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1610000000000001e-06 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919851+0000", 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919879+0000", 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.756e-05 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919896+0000", 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6600999999999999e-05 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919971+0000", 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5301000000000003e-05 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.919997+0000", 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5851e-05 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924043+0000", 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040459479999999997 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924086+0000", 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3022000000000002e-05 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.949603527000001, 2024-05-08T08:25:10.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073213000000000002, 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920654+0000", 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920686+0000", 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2338000000000001e-05 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920695+0000", 2024-05-08T08:25:10.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5650000000000007e-06 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920701+0000", 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8170000000000004e-06 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920749+0000", 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8071000000000001e-05 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920768+0000", 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8546000000000002e-05 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927915+0000", 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071468269999999997 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927974+0000", 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9648000000000002e-05 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.949076905, 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068250300000000002, 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:10.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-07 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921185+0000", 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.135e-06 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921195+0000", 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921201+0000", 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921321+0000", 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011977 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921339+0000", 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7608000000000001e-05 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927977+0000", 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066382029999999996 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928005+0000", 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7722e-05 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.949009379, 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067827720000000003, 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921248+0000", 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:25:10.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921252+0000", 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4449999999999998e-06 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921261+0000", 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5830000000000007e-06 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921267+0000", 2024-05-08T08:25:10.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921577+0000", 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031014399999999999 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921595+0000", 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7802e-05 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928006+0000", 2024-05-08T08:25:10.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064114100000000002 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928030+0000", 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4042000000000001e-05 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:109 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'64)", 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.948430738000001, 2024-05-08T08:25:10.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062286720000000002, 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.44e-07 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921855+0000", 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8645000000000001e-05 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921865+0000", 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7310000000000005e-06 2024-05-08T08:25:10.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921871+0000", 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921924+0000", 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3109999999999998e-05 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921945+0000", 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1118e-05 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928031+0000", 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060861200000000004 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928055+0000", 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3141000000000001e-05 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:110 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'64)", 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.947958700999999, 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057807739999999998, 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922304+0000", 2024-05-08T08:25:10.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.288e-05 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922323+0000", 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3749999999999999e-06 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922387+0000", 2024-05-08T08:25:10.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3932999999999999e-05 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922407+0000", 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9885e-05 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928056+0000", 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056492180000000001 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928079+0000", 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2467e-05 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:111 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'64)", 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.947612886, 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005461502, 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-05-08T08:25:10.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922649+0000", 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922658+0000", 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8419999999999994e-06 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922664+0000", 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1290000000000001e-06 2024-05-08T08:25:10.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922725+0000", 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922743+0000", 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8142999999999999e-05 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928080+0000", 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053372430000000002 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928105+0000", 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5004999999999999e-05 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'64)", 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.947295574, 2024-05-08T08:25:10.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055842340000000004, 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922962+0000", 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923087+0000", 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012524199999999999 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923097+0000", 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8700000000000004e-06 2024-05-08T08:25:10.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923103+0000", 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2049999999999999e-06 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923163+0000", 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0554000000000003e-05 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923182+0000", 2024-05-08T08:25:10.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8515000000000001e-05 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928508+0000", 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053261330000000003 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928545+0000", 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7165000000000001e-05 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:113 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'64)", 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.946717782, 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050342659999999999, 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0200000000000002e-07 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923547+0000", 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2119999999999997e-06 2024-05-08T08:25:10.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923556+0000", 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3470000000000005e-06 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923564+0000", 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8779999999999998e-06 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923625+0000", 2024-05-08T08:25:10.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1309999999999994e-05 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923644+0000", 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8403999999999999e-05 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928547+0000", 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049035349999999997 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928573+0000", 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5978000000000001e-05 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:114 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'76, mlcod=16'64)", 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.923916+0000", 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.946340472999999, 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046851849999999997, 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.920 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923916+0000", 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923916+0000", 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923917+0000", 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6400000000000002e-07 2024-05-08T08:25:10.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923922+0000", 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923934+0000", 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1206000000000001e-05 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923940+0000", 2024-05-08T08:25:10.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.088e-06 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924190+0000", 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025044899999999997 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924208+0000", 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8008000000000001e-05 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928577+0000", 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043690730000000002 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928601+0000", 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4207e-05 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:115 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'77, mlcod=16'64)", 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.945908019999999, 2024-05-08T08:25:10.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042775119999999998, 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924349+0000", 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5499999999999997e-07 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924418+0000", 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8512999999999999e-05 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924430+0000", 2024-05-08T08:25:10.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2037e-05 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924437+0000", 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3019999999999997e-06 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924482+0000", 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4697999999999998e-05 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.924500+0000", 2024-05-08T08:25:10.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7666999999999999e-05 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928603+0000", 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041028770000000004 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928626+0000", 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3563000000000001e-05 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:143 2.0 e17/15 2:30306672:devicehealth::main.db.0000000000000000:head v 17'101, mlcod=17'98)", 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:58.627220+0000", 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.243036721999999, 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004400101, 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627220+0000", 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627220+0000", 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627221+0000", 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3099999999999997e-07 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627283+0000", 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2219999999999997e-05 2024-05-08T08:25:10.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627294+0000", 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1521e-05 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627300+0000", 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8930000000000002e-06 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627342+0000", 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2255000000000001e-05 2024-05-08T08:25:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.627355+0000", 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-05 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.631580+0000", 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042257320000000003 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:58.631620+0000", 2024-05-08T08:25:10.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9739e-05 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1599 3.0 3:01280103:::rbd_data.119116cc7dd0.0000000000000180:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.046640+0000", 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.82361677799999999, 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046157560000000004, 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1599 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.046640+0000", 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.046640+0000", 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.046641+0000", 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8899999999999998e-07 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.046646+0000", 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0980000000000001e-06 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.046656+0000", 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0176999999999999e-05 2024-05-08T08:25:10.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.046665+0000", 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.969e-06 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.046973+0000", 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00030773600000000002 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.047054+0000", 2024-05-08T08:25:10.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1287999999999998e-05 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.047172+0000", 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000117464 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.048984+0000", 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001812564 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.051166+0000", 2024-05-08T08:25:10.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021817410000000001 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.051217+0000", 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1502000000000003e-05 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.051256+0000", 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8228000000000001e-05 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1619 3.19 3:9ff62921:::rbd_data.119116cc7dd0.000000000000022a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.054401+0000", 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.81585591199999996, 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047229780000000001, 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1619 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054401+0000", 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054401+0000", 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054402+0000", 2024-05-08T08:25:10.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2950000000000001e-06 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054407+0000", 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4480000000000002e-06 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054421+0000", 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3876000000000001e-05 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054429+0000", 2024-05-08T08:25:10.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0430000000000008e-06 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054486+0000", 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6418000000000001e-05 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054575+0000", 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9190000000000005e-05 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:10.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.054701+0000", 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012551700000000001 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.056474+0000", 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017739159999999999 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.059049+0000", 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025743350000000001 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.059088+0000", 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9337e-05 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.059124+0000", 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5602999999999998e-05 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1825 3.4 3:23b8ebb3:::rbd_data.119116cc7dd0.00000000000001e0:head [set-alloc-hint object_size 262144 write_size 262144,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.251364+0000", 2024-05-08T08:25:10.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.61889225199999998, 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004867405, 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1825 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251364+0000", 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251364+0000", 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251366+0000", 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3820000000000001e-06 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251378+0000", 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1841999999999999e-05 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251394+0000", 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6398000000000001e-05 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251402+0000", 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4179999999999997e-06 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251457+0000", 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249000000000002e-05 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251504+0000", 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7549000000000002e-05 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251582+0000", 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7501000000000002e-05 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.252660+0000", 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010780970000000001 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256143+0000", 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003483089 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256184+0000", 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1291000000000002e-05 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256232+0000", 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589000000000001e-05 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2153 3.8 e23/18 3:11eb6c23:::rbd_data.119116cc7dd0.000000000000039a:head v 23'33, mlcod=23'31)", 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.547715+0000", 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.32254119599999997, 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046911139999999997, 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:10.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547715+0000", 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547715+0000", 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:10.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547717+0000", 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.125e-06 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547722+0000", 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9279999999999998e-06 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547735+0000", 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2513e-05 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:10.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547742+0000", 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7510000000000002e-06 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547822+0000", 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0063999999999998e-05 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547842+0000", 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0119000000000002e-05 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552364+0000", 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045221059999999997 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552407+0000", 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2508e-05 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:10.954 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:10.956 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:10.956 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:10.963+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:11.057 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:11.065+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:11.066 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:11.066 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:11.066 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:11.075 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:11.157 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:11.165+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:11.184 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:11.184 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:11.184 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:11.193 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:11.259 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:11.266+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1729 3.9 e23/18 3:9695f221:::rbd_data.119116cc7dd0.00000000000002df:head v 23'70, mlcod=23'68)", 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.178435874, 2024-05-08T08:25:11.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001912813, 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171947+0000", 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2610000000000001e-06 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171962+0000", 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4827999999999999e-05 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171983+0000", 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1542000000000001e-05 2024-05-08T08:25:11.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172004+0000", 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0577e-05 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172072+0000", 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8205000000000002e-05 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172097+0000", 2024-05-08T08:25:11.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5004999999999999e-05 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173793+0000", 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001696513 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173858+0000", 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4882000000000007e-05 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1738 3.0 e23/18 3:071e2661:::rbd_data.119116cc7dd0.0000000000000012:head v 23'35, mlcod=23'33)", 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.179660+0000", 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.1707211959999999, 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00095831200000000001, 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179660+0000", 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179660+0000", 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179661+0000", 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.28e-06 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179691+0000", 2024-05-08T08:25:11.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9887999999999999e-05 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179710+0000", 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.895e-05 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179720+0000", 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0131e-05 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179787+0000", 2024-05-08T08:25:11.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.179814+0000", 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7186999999999999e-05 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.180568+0000", 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075401000000000005 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.180618+0000", 2024-05-08T08:25:11.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0257999999999999e-05 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1752 3.0 e23/18 3:04522806:::rbd_data.119116cc7dd0.000000000000001a:head v 23'36, mlcod=23'34)", 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.195775+0000", 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.154606206, 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00089703099999999996, 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195775+0000", 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195775+0000", 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195776+0000", 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1069999999999999e-06 2024-05-08T08:25:11.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195799+0000", 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3217000000000001e-05 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195819+0000", 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9981000000000001e-05 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195828+0000", 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.85e-06 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195902+0000", 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3819999999999995e-05 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.195929+0000", 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7080000000000002e-05 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.196630+0000", 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070034600000000002 2024-05-08T08:25:11.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.196672+0000", 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2629999999999997e-05 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1765 3.4 e23/18 3:21a1f84d:::rbd_data.119116cc7dd0.0000000000000177:head v 23'25, mlcod=22'23)", 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.203281+0000", 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.147100555, 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093612100000000005, 2024-05-08T08:25:11.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203281+0000", 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203281+0000", 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203282+0000", 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8400000000000003e-07 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203304+0000", 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1933000000000002e-05 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203321+0000", 2024-05-08T08:25:11.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7638e-05 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203330+0000", 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7819999999999999e-06 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203404+0000", 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4342000000000001e-05 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.203427+0000", 2024-05-08T08:25:11.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3105e-05 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.204176+0000", 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00074871400000000004 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.204217+0000", 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0723000000000003e-05 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1805 3.4 e23/18 3:247a66e4:::rbd_data.119116cc7dd0.0000000000000088:head v 23'26, mlcod=22'24)", 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.1169626509999999, 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002584514, 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233420+0000", 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7100000000000011e-07 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233438+0000", 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8084000000000002e-05 2024-05-08T08:25:11.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233454+0000", 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5851000000000001e-05 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233463+0000", 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9819999999999997e-06 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233527+0000", 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4140999999999995e-05 2024-05-08T08:25:11.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233551+0000", 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4606000000000001e-05 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.235949+0000", 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002397475 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236003+0000", 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4404000000000003e-05 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1816 3.4 e23/18 3:22bef275:::rbd_data.119116cc7dd0.0000000000000147:head v 23'27, mlcod=23'25)", 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.1046551019999999, 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012053820000000001, 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245727+0000", 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0699999999999999e-06 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245745+0000", 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7932999999999999e-05 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245763+0000", 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8139999999999999e-05 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245772+0000", 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2789999999999998e-06 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245849+0000", 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7167000000000002e-05 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245876+0000", 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7260999999999999e-05 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.246893+0000", 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001017015 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.246932+0000", 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8516999999999997e-05 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1825 3.4 e23/18 3:23b8ebb3:::rbd_data.119116cc7dd0.00000000000001e0:head v 23'28, mlcod=23'26)", 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.251654+0000", 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0987278629999999, 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00088948499999999997, 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251654+0000", 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251654+0000", 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251655+0000", 2024-05-08T08:25:11.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0359999999999999e-06 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251671+0000", 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.662e-05 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251687+0000", 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6103000000000002e-05 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251696+0000", 2024-05-08T08:25:11.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1179999999999998e-06 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251775+0000", 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8786000000000001e-05 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251802+0000", 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6874000000000001e-05 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.252510+0000", 2024-05-08T08:25:11.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070779800000000004 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.252543+0000", 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3149999999999999e-05 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1829 3.4 e23/18 3:243982ca:::rbd_data.119116cc7dd0.00000000000001e4:head v 23'29, mlcod=23'27)", 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0936701499999999, 2024-05-08T08:25:11.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002557368, 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256712+0000", 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1680000000000001e-06 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256722+0000", 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3910000000000004e-06 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256732+0000", 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0682999999999999e-05 2024-05-08T08:25:11.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256738+0000", 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256783+0000", 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5015000000000002e-05 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256796+0000", 2024-05-08T08:25:11.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3745e-05 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259238+0000", 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024414409999999999 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259269+0000", 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0648999999999997e-05 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1830 3.0 e23/18 3:06700602:::rbd_data.119116cc7dd0.00000000000001e5:head v 23'37, mlcod=23'35)", 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.259595+0000", 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0907861160000001, 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00081104400000000002, 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259595+0000", 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259595+0000", 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259596+0000", 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999999e-07 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259616+0000", 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0276999999999999e-05 2024-05-08T08:25:11.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259629+0000", 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2481e-05 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259637+0000", 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6600000000000001e-06 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259692+0000", 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4755e-05 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259714+0000", 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2126000000000001e-05 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.260368+0000", 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065397999999999997 2024-05-08T08:25:11.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.260406+0000", 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7966000000000002e-05 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1838 3.9 e23/18 3:9224efc6:::rbd_data.119116cc7dd0.0000000000000253:head v 23'72, mlcod=23'70)", 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.266165+0000", 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.084216748, 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00092662899999999999, 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266165+0000", 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266165+0000", 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266166+0000", 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7999999999999993e-07 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266190+0000", 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4273999999999998e-05 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266202+0000", 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2062000000000001e-05 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266209+0000", 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7889999999999997e-06 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266262+0000", 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3576999999999999e-05 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.266286+0000", 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3360999999999999e-05 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.267040+0000", 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075420999999999995 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.267091+0000", 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1375999999999997e-05 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1903 3.4 e23/18 3:24e0ebf3:::rbd_data.119116cc7dd0.0000000000000376:head v 23'30, mlcod=23'28)", 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0245065440000001, 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015443570000000001, 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325876+0000", 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0020000000000001e-06 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325906+0000", 2024-05-08T08:25:11.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9688000000000001e-05 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325920+0000", 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4558e-05 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325927+0000", 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5459999999999997e-06 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325980+0000", 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3029999999999999e-05 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325996+0000", 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5988999999999999e-05 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.327370+0000", 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001374632 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.327419+0000", 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8912000000000002e-05 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1943 3.4 e23/18 3:24e0ebf3:::rbd_data.119116cc7dd0.0000000000000376:head v 23'31, mlcod=23'29)", 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.372274+0000", 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.97810714099999996, 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001109014, 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.392 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372274+0000", 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372274+0000", 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372275+0000", 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.186e-06 2024-05-08T08:25:11.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372284+0000", 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0189999999999995e-06 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372305+0000", 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0928e-05 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372316+0000", 2024-05-08T08:25:11.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1080999999999999e-05 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372383+0000", 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6982000000000003e-05 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.372404+0000", 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0242e-05 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373340+0000", 2024-05-08T08:25:11.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093635400000000003 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373383+0000", 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3222000000000001e-05 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2041 3.9 e23/18 3:9250a055:::rbd_data.119116cc7dd0.000000000000027d:head v 23'85, mlcod=23'83)", 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.453469+0000", 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.896911876, 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083489999999999997, 2024-05-08T08:25:11.396 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453469+0000", 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453469+0000", 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453470+0000", 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5099999999999998e-07 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453492+0000", 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.215e-05 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453508+0000", 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5058000000000001e-05 2024-05-08T08:25:11.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453515+0000", 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3339999999999996e-06 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453581+0000", 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5637999999999997e-05 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.453601+0000", 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.001e-05 2024-05-08T08:25:11.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.454253+0000", 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00065265799999999997 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.454304+0000", 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1201e-05 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2088 3.4 e23/18 3:206618ae:::rbd_data.119116cc7dd0.0000000000000195:head v 23'37, mlcod=23'35)", 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.86329219099999999, 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001240746, 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487090+0000", 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.189e-06 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487103+0000", 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2685e-05 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487121+0000", 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8134e-05 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487130+0000", 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.833e-06 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487193+0000", 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2898999999999995e-05 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487210+0000", 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6946e-05 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.488287+0000", 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001077089 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.488330+0000", 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2970999999999999e-05 2024-05-08T08:25:11.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2113 3.4 e23/18 3:23bb875f:::rbd_data.119116cc7dd0.00000000000000a6:head v 23'38, mlcod=23'36)", 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.84019876299999996, 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020550019999999998, 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510184+0000", 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.387e-06 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510206+0000", 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2084999999999999e-05 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510226+0000", 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0046000000000001e-05 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510238+0000", 2024-05-08T08:25:11.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1843e-05 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510297+0000", 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8959999999999998e-05 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510313+0000", 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5787999999999999e-05 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512193+0000", 2024-05-08T08:25:11.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001879999 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512238+0000", 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4894000000000001e-05 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2121 3.4 e23/18 3:27cd253f:::rbd_data.119116cc7dd0.0000000000000156:head v 23'39, mlcod=23'37)", 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.834624111, 2024-05-08T08:25:11.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023924910000000001, 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515758+0000", 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999996e-07 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515777+0000", 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8817999999999998e-05 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515792+0000", 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5277999999999998e-05 2024-05-08T08:25:11.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515800+0000", 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6000000000000001e-06 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515868+0000", 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7834000000000001e-05 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515899+0000", 2024-05-08T08:25:11.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1473999999999997e-05 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.518104+0000", 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022054000000000002 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.518150+0000", 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5318000000000002e-05 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2137 3.4 e23/18 3:206618ae:::rbd_data.119116cc7dd0.0000000000000195:head v 23'41, mlcod=23'38)", 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.80887628700000003, 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027337440000000002, 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541506+0000", 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-07 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541511+0000", 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5550000000000002e-06 2024-05-08T08:25:11.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541523+0000", 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.207e-05 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541530+0000", 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541599+0000", 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8143000000000001e-05 2024-05-08T08:25:11.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541617+0000", 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8794999999999999e-05 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544188+0000", 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025705290000000002 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544239+0000", 2024-05-08T08:25:11.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0967000000000002e-05 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2141 3.9 e23/18 3:9250a055:::rbd_data.119116cc7dd0.000000000000027d:head v 23'88, mlcod=23'86)", 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.805900587, 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023341719999999998, 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544482+0000", 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544487+0000", 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1590000000000001e-06 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544499+0000", 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2621e-05 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544506+0000", 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7510000000000002e-06 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544577+0000", 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0801000000000002e-05 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544599+0000", 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1642e-05 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546771+0000", 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021727180000000001 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546815+0000", 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3683000000000002e-05 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2207 3.9 e23/18 3:961d4b12:::rbd_data.119116cc7dd0.00000000000000cf:head v 23'91, mlcod=23'89)", 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.74894472199999995, 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001369622, 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601438+0000", 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6840000000000001e-06 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601456+0000", 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7320999999999999e-05 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601475+0000", 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9448000000000001e-05 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601484+0000", 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601554+0000", 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9820000000000006e-05 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601581+0000", 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6517e-05 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.602759+0000", 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001178183 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.602806+0000", 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7311000000000002e-05 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2229 3.0 e23/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 23'39, mlcod=23'37)", 2024-05-08T08:25:11.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.624262+0000", 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.72611964799999995, 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010159430000000001, 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624262+0000", 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624262+0000", 2024-05-08T08:25:11.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624263+0000", 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8700000000000004e-07 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624274+0000", 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1671999999999999e-05 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624290+0000", 2024-05-08T08:25:11.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5142e-05 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624297+0000", 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.34e-06 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624368+0000", 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1598000000000001e-05 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.624397+0000", 2024-05-08T08:25:11.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8742999999999999e-05 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.625234+0000", 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00083662600000000004 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.625278+0000", 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3835000000000003e-05 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:11.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:11.431 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:11.432 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:11.433 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:11.439+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:11.533 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:11.540+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:11.583 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:11.583 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:11.583 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:11.583 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:11.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:11.633 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:11.640+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:11.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:25:11.697 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone1 from 0xe2da07c (overlap 0x84e204) to 0x352734 2024-05-08T08:25:11.734 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:11.741+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:11.837 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:11.837 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:11.837 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:11.837 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:11.838 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:11.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:11.838 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-05-08T08:25:11.839 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:11.845+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:11.931 INFO:teuthology.orchestra.run.smithi031.stdout:380 read 0xbc7e395 thru 0xbc8618c (0x7df8 bytes) 2024-05-08T08:25:11.932 INFO:teuthology.orchestra.run.smithi031.stdout:381 write 0xc59a5be thru 0xc59fd33 (0x5776 bytes) 2024-05-08T08:25:11.938 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:11.946+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:11.945 INFO:teuthology.orchestra.run.smithi031.stdout:383 read 0x98c7282 thru 0x98c81f0 (0xf6f bytes) 2024-05-08T08:25:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:384 read 0x65b0f3c thru 0x65b45b9 (0x367e bytes) 2024-05-08T08:25:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:386 read 0x81f01af thru 0x81ff5a6 (0xf3f8 bytes) 2024-05-08T08:25:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:387 read 0xbb478d4 thru 0xbb552ac (0xd9d9 bytes) 2024-05-08T08:25:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:388 read 0x66e1c6e thru 0x66ed463 (0xb7f6 bytes) 2024-05-08T08:25:11.949 INFO:teuthology.orchestra.run.smithi031.stdout:390 trunc from 0xe2da07c to 0xbb24525 2024-05-08T08:25:11.970 INFO:teuthology.orchestra.run.smithi031.stdout:391 read 0x171de84 thru 0x1723582 (0x56ff bytes) 2024-05-08T08:25:11.970 INFO:teuthology.orchestra.run.smithi031.stdout:392 write 0x1b643d5 thru 0x1b6b028 (0x6c54 bytes) 2024-05-08T08:25:11.991 INFO:teuthology.orchestra.run.smithi031.stdout:394 punch from 0x5cf9c09 to 0x5d01109, (0x7500 bytes) 2024-05-08T08:25:11.993 INFO:teuthology.orchestra.run.smithi031.stdout:395 trunc from 0xbb24525 to 0xd4dd57b 2024-05-08T08:25:12.002 INFO:teuthology.orchestra.run.smithi031.stdout:397 write 0xbf24900 thru 0xbf30f8c (0xc68d bytes) 2024-05-08T08:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:12.004 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1666 3.b 3:d623a406:::rbd_data.119116cc7dd0.000000000000025e:head [set-alloc-hint object_size 262144 write_size 262144,write 65536~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.105417+0000", 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.9046866040000001, 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035096160000000001, 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1666 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105417+0000", 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105417+0000", 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105419+0000", 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7239999999999999e-06 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105433+0000", 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4338000000000001e-05 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105449+0000", 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6050999999999999e-05 2024-05-08T08:25:12.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105456+0000", 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1860000000000003e-06 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105524+0000", 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7989000000000002e-05 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105614+0000", 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0093000000000006e-05 2024-05-08T08:25:12.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.105766+0000", 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000151862 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.106611+0000", 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084506200000000005 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.108819+0000", 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022078200000000001 2024-05-08T08:25:12.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.108875+0000", 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5627999999999997e-05 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.108927+0000", 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1863000000000001e-05 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1684 3.7 3:e31dbc09:::rbd_data.119116cc7dd0.0000000000000188:head [set-alloc-hint object_size 262144 write_size 262144,write 149504~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.121640+0000", 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8884634679999999, 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054458309999999999, 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1684 2024-05-08T08:25:12.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121640+0000", 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121640+0000", 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121642+0000", 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4920000000000001e-06 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121657+0000", 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5727999999999999e-05 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121674+0000", 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6531000000000001e-05 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121683+0000", 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1889999999999996e-06 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121757+0000", 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3639000000000004e-05 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121849+0000", 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2428000000000003e-05 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.122005+0000", 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155921 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.122753+0000", 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000748374 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.126964+0000", 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042103540000000004 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127031+0000", 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7693000000000004e-05 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127086+0000", 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4481999999999998e-05 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1691 3.7 3:e31dbc09:::rbd_data.119116cc7dd0.0000000000000188:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.137900+0000", 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8722039050000001, 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004191742, 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1691 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.137900+0000", 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.137900+0000", 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.137902+0000", 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9599999999999999e-06 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.137910+0000", 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3650000000000004e-06 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.137930+0000", 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0520999999999999e-05 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.137945+0000", 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4267000000000001e-05 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.138026+0000", 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1578999999999999e-05 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.138120+0000", 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3708999999999993e-05 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.138239+0000", 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011861700000000001 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.138921+0000", 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068226699999999996 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.142000+0000", 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030788719999999999 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.142059+0000", 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9049000000000003e-05 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.142091+0000", 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2536000000000002e-05 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1726 3.b 3:d54f3c11:::rbd_data.119116cc7dd0.00000000000002dc:head [set-alloc-hint object_size 262144 write_size 262144,write 210944~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.168544+0000", 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.8415598800000001, 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056432180000000002, 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1726 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168544+0000", 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168544+0000", 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168545+0000", 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.668e-06 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168559+0000", 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3949e-05 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168576+0000", 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6396e-05 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168583+0000", 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6129999999999998e-06 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168639+0000", 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5265e-05 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168758+0000", 2024-05-08T08:25:12.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119265 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168870+0000", 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011209699999999999 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.170243+0000", 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001373256 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174099+0000", 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038553760000000002 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174142+0000", 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3720000000000002e-05 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174187+0000", 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4613000000000001e-05 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.028 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1925 3.7 3:e623465e:::rbd_data.119116cc7dd0.00000000000003a1:head [set-alloc-hint object_size 262144 write_size 262144,write 59392~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.346501+0000", 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.6636027019999999, 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054341750000000003, 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1925 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346501+0000", 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346501+0000", 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346502+0000", 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3099999999999996e-07 2024-05-08T08:25:12.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346509+0000", 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346519+0000", 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0274e-05 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346525+0000", 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5509999999999999e-06 2024-05-08T08:25:12.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346579+0000", 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3998e-05 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346629+0000", 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0670000000000001e-05 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346716+0000", 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6349999999999998e-05 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.347249+0000", 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053301200000000003 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.351865+0000", 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046158220000000003 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.351909+0000", 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4480000000000001e-05 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.351935+0000", 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5812999999999998e-05 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1950 3.7 3:e623465e:::rbd_data.119116cc7dd0.00000000000003a1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.373087+0000", 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.637016016, 2024-05-08T08:25:12.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050660879999999998, 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1950 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373087+0000", 2024-05-08T08:25:12.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373087+0000", 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373088+0000", 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1999999999999999e-07 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373092+0000", 2024-05-08T08:25:12.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373101+0000", 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9080000000000007e-06 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373107+0000", 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.981e-06 2024-05-08T08:25:12.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373166+0000", 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8687000000000002e-05 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373255+0000", 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8771999999999996e-05 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373382+0000", 2024-05-08T08:25:12.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012696300000000001 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.374231+0000", 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084901499999999995 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.378081+0000", 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038498199999999999 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.378110+0000", 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9611000000000002e-05 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.378154+0000", 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3306999999999997e-05 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1983 3.7 3:e216bce7:::rbd_data.119116cc7dd0.00000000000000ae:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.6002249549999998, 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052937829999999998, 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1983 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:12.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8100000000000001e-07 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409890+0000", 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0968000000000001e-05 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409903+0000", 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2294e-05 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409908+0000", 2024-05-08T08:25:12.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-06 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409950+0000", 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2185999999999999e-05 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409993+0000", 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2233e-05 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410050+0000", 2024-05-08T08:25:12.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7778999999999997e-05 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410519+0000", 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046863200000000002 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415079+0000", 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045602949999999998 2024-05-08T08:25:12.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415132+0000", 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2516000000000003e-05 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415172+0000", 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0528999999999998e-05 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1984 3.7 3:e30db22a:::rbd_data.119116cc7dd0.00000000000000af:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.410668+0000", 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5994358769999999, 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046636239999999999, 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1984 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410668+0000", 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410668+0000", 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410668+0000", 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3399999999999998e-07 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410672+0000", 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.765e-06 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410683+0000", 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1351e-05 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410691+0000", 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3900000000000004e-06 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410732+0000", 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1477999999999998e-05 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410782+0000", 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9469999999999999e-05 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410841+0000", 2024-05-08T08:25:12.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8891000000000002e-05 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.411396+0000", 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00055520000000000005 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415229+0000", 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003833496 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415287+0000", 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7787000000000001e-05 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415331+0000", 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4061999999999999e-05 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1999 3.1e 3:7e7612a5:::rbd_data.119116cc7dd0.0000000000000179:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.416389+0000", 2024-05-08T08:25:12.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5937144779999999, 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046829560000000003, 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1999 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416389+0000", 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416389+0000", 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416390+0000", 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2699999999999999e-06 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416395+0000", 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416408+0000", 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2519e-05 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416415+0000", 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9219999999999997e-06 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416472+0000", 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6471999999999997e-05 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416540+0000", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8308999999999999e-05 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416636+0000", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5778000000000004e-05 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.417097+0000", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046121999999999998 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.420976+0000", 2024-05-08T08:25:12.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038786670000000001 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.421025+0000", 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8607999999999999e-05 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.421072+0000", 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7451999999999999e-05 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2111 3.18 3:1d87061b:::rbd_data.119116cc7dd0.00000000000000a4:head [set-alloc-hint object_size 262144 write_size 262144,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.506882+0000", 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.5032216819999999, 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055828070000000004, 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2111 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506882+0000", 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506882+0000", 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506883+0000", 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.457e-06 2024-05-08T08:25:12.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506903+0000", 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9465000000000001e-05 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506919+0000", 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6268000000000001e-05 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506928+0000", 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.738e-06 2024-05-08T08:25:12.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506980+0000", 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2748999999999999e-05 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.507079+0000", 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8148999999999998e-05 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.507167+0000", 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8405999999999993e-05 2024-05-08T08:25:12.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.509392+0000", 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022244750000000001 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512382+0000", 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029903820000000002 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512418+0000", 2024-05-08T08:25:12.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6390000000000002e-05 2024-05-08T08:25:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512465+0000", 2024-05-08T08:25:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6328000000000001e-05 2024-05-08T08:25:12.066 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2140 3.5 e23/18 3:a1b38341:::rbd_data.119116cc7dd0.000000000000027c:head v 23'19, mlcod=23'17)", 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.542251+0000", 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4678523110000001, 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044467780000000002, 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542251+0000", 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542251+0000", 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542252+0000", 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5899999999999995e-07 2024-05-08T08:25:12.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542260+0000", 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6930000000000007e-06 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542275+0000", 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5175e-05 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542284+0000", 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2e-06 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542332+0000", 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7890999999999999e-05 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.542346+0000", 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3891e-05 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546630+0000", 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042845230000000002 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546698+0000", 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7645999999999996e-05 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2148 3.b 3:d6d99b55:::rbd_data.119116cc7dd0.0000000000000329:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4630655619999999, 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055972469999999996, 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.071 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2148 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.13e-07 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547042+0000", 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.585e-06 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547050+0000", 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.089e-06 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547058+0000", 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9060000000000002e-06 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547541+0000", 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048342399999999998 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547616+0000", 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4493000000000007e-05 2024-05-08T08:25:12.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547717+0000", 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100739 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552422+0000", 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047050369999999996 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552546+0000", 2024-05-08T08:25:12.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012416599999999999 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552594+0000", 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8498000000000002e-05 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552635+0000", 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0797000000000003e-05 2024-05-08T08:25:12.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2147 3.15 e23/18 3:aace63c7:::rbd_data.119116cc7dd0.0000000000000328:head v 23'16, mlcod=23'14)", 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4627538790000001, 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051080810000000004, 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1600000000000001e-07 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547357+0000", 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7660000000000001e-06 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547367+0000", 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0142e-05 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547374+0000", 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8970000000000002e-06 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547427+0000", 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2701999999999998e-05 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547446+0000", 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.946e-05 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552415+0000", 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004969238 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552458+0000", 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2153 3.8 3:11eb6c23:::rbd_data.119116cc7dd0.000000000000039a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.547373+0000", 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.4627305879999999, 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052376890000000002, 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2153 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547373+0000", 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547373+0000", 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547374+0000", 2024-05-08T08:25:12.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7499999999999999e-07 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547378+0000", 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4329999999999997e-06 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547390+0000", 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1804000000000001e-05 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547398+0000", 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7819999999999994e-06 2024-05-08T08:25:12.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547456+0000", 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8224999999999999e-05 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547530+0000", 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3781000000000004e-05 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547640+0000", 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110091 2024-05-08T08:25:12.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552462+0000", 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048225810000000003 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552549+0000", 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6224999999999995e-05 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552585+0000", 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5880000000000002e-05 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552611+0000", 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6012000000000001e-05 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2334 3.1e 3:7c0c4f5a:::rbd_data.119116cc7dd0.0000000000000017:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:12.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.30260784499999999, 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048629379999999998, 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2334 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.090 INFO:teuthology.orchestra.run.smithi031.stdout:398 read 0xa02e98b thru 0xa037d0b (0x9381 bytes) 2024-05-08T08:25:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707500+0000", 2024-05-08T08:25:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8399999999999997e-06 2024-05-08T08:25:12.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707509+0000", 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5709999999999998e-06 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707515+0000", 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707572+0000", 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707653+0000", 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0995999999999995e-05 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707826+0000", 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017260100000000001 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709844+0000", 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020187820000000002 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.712250+0000", 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002405917 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.712310+0000", 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9596999999999999e-05 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.712359+0000", 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8634999999999998e-05 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2371 3.8 3:158ebef6:::rbd_data.119116cc7dd0.0000000000000204:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.28941999299999999, 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094009569999999997, 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.094 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2371 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5099999999999998e-07 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720689+0000", 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8099999999999997e-06 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720701+0000", 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.153e-05 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720710+0000", 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3640000000000005e-06 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720778+0000", 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7866000000000004e-05 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720864+0000", 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6384999999999994e-05 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721113+0000", 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024862300000000001 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725338+0000", 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042250090000000001 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729983+0000", 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046450659999999998 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.730040+0000", 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6422999999999998e-05 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.730084+0000", 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4929999999999998e-05 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2372 3.7 3:e6a28cad:::rbd_data.119116cc7dd0.0000000000000205:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721541+0000", 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.288562604, 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056699929999999999, 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2372 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721541+0000", 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721541+0000", 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721542+0000", 2024-05-08T08:25:12.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.37e-07 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721546+0000", 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2690000000000001e-06 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721558+0000", 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1883e-05 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721565+0000", 2024-05-08T08:25:12.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721640+0000", 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4742999999999999e-05 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721718+0000", 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8545000000000001e-05 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721894+0000", 2024-05-08T08:25:12.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000175954 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.723729+0000", 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001834303 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727129+0000", 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034002059999999998 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727166+0000", 2024-05-08T08:25:12.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7283999999999997e-05 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727211+0000", 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4793999999999998e-05 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2375 3.b 3:d52caae0:::rbd_data.119116cc7dd0.0000000000000208:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721944+0000", 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.28815985500000002, 2024-05-08T08:25:12.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053243179999999998, 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2375 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721944+0000", 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721944+0000", 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721945+0000", 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.64e-07 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721949+0000", 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4989999999999996e-06 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721961+0000", 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721972+0000", 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0436e-05 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722039+0000", 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7657999999999995e-05 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722115+0000", 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5370000000000005e-05 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722275+0000", 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015985000000000001 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725627+0000", 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033519410000000002 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727159+0000", 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001531969 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727212+0000", 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3507000000000001e-05 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727268+0000", 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5892e-05 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2377 3.8 3:17a99b47:::rbd_data.119116cc7dd0.000000000000038a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.722517+0000", 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.28758677999999999, 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01043969, 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2377 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722517+0000", 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722517+0000", 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722518+0000", 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4099999999999997e-07 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722523+0000", 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722534+0000", 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0994e-05 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722541+0000", 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3599999999999998e-06 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722580+0000", 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9405e-05 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722642+0000", 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1231999999999999e-05 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722762+0000", 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000120426 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.730110+0000", 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073483749999999999 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732831+0000", 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002720685 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732897+0000", 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6106999999999996e-05 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732956+0000", 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9268999999999998e-05 2024-05-08T08:25:12.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2641 3.7 3:e564731b:::rbd_object_map.1191a2472683:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:12.002123+0000", 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.0079801239999999999, 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043016929999999997, 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:12.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2641 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002123+0000", 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002123+0000", 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002126+0000", 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8040000000000002e-06 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002140+0000", 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3481e-05 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002165+0000", 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4998e-05 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002179+0000", 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3905e-05 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002253+0000", 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4589e-05 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002332+0000", 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8435000000000004e-05 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:12.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.002521+0000", 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018969200000000001 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.003234+0000", 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00071223499999999997 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.006360+0000", 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031262600000000001 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:12.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.006393+0000", 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.362e-05 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.006425+0000", 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1674000000000002e-05 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:12.122 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:12.123 INFO:teuthology.orchestra.run.smithi031.stdout:399 write 0x5429ff6 thru 0x542eca1 (0x4cac bytes) 2024-05-08T08:25:12.124 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:400 write 0x5b7d70a thru 0x5b84cfe (0x75f5 bytes) 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:402 punch from 0xb9c8cfe to 0xb9d1cf9, (0x8ffb bytes) 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:403 punch from 0x3fb31b4 to 0x3fbb140, (0x7f8c bytes) 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:404 read 0x646b533 thru 0x64730c2 (0x7b90 bytes) 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:405 writesame 0x5a64add thru 0x5a6e2bd data_size 0x1(0x97e1 bytes) 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:406 read 0x1fcde67 thru 0x1fd54af (0x7649 bytes) 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:407 writesame 0xc359dad thru 0xc35ec30 data_size 0x64(0x4e84 bytes) 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:12.126 INFO:teuthology.orchestra.run.smithi031.stdout:409 write 0x7e4b104 thru 0x7e4fb12 (0x4a0f bytes) 2024-05-08T08:25:12.127 INFO:teuthology.orchestra.run.smithi031.stdout:410 read 0x722dec2 thru 0x723b418 (0xd557 bytes) 2024-05-08T08:25:12.127 INFO:teuthology.orchestra.run.smithi031.stdout:411 clone 3 order 18 su 32768 sc 8 2024-05-08T08:25:12.127 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:12.133+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:12.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:12.227 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:12.234+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:12.232 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:12.232 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:12.233 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:12.242 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:12.293 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:12.294 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:12.311 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:12.312 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:12.328 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:12.335+0000 7f890a466640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32761) UID: 0 2024-05-08T08:25:12.328 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:12.329 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:12.346 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:12.346 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:12.349 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:12.349 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:12.349 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:12.349 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:12.349 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:12.358 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:12.364 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:12.364 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:12.382 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:12.382 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:12.401 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:12.401 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:12.418 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:12.419 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:12.429 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:12.436+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:12.465 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:12.465 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:12.474 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:12.557 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:12.565+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:12.622 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:12.622 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:12.630 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:12.638+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:12.631 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:12.633 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:25:12.696 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone2 intact 2024-05-08T08:25:12.731 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:12.738+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:12.777 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:12.777 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:12.785 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:12.832 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:12.839+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:12.848 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-05-08T08:25:12.858 INFO:teuthology.orchestra.run.smithi031.stdout:414 write 0x6f12047 thru 0x6f15337 (0x32f1 bytes) 2024-05-08T08:25:12.871 INFO:teuthology.orchestra.run.smithi031.stdout:415 trunc from 0xd4dd57b to 0x977adbd 2024-05-08T08:25:12.889 INFO:teuthology.orchestra.run.smithi031.stdout:417 punch from 0xf55652 to 0xf64a10, (0xf3be bytes) 2024-05-08T08:25:12.889 INFO:teuthology.orchestra.run.smithi031.stdout:418 read 0x5d83019 thru 0x5d8a437 (0x741f bytes) 2024-05-08T08:25:12.895 INFO:teuthology.orchestra.run.smithi031.stdout:419 write 0xa171a42 thru 0xa17e0a6 (0xc665 bytes) 2024-05-08T08:25:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:420 read 0x3898438 thru 0x389c5f4 (0x41bd bytes) 2024-05-08T08:25:12.898 INFO:teuthology.orchestra.run.smithi031.stdout:421 write 0x9882c85 thru 0x98894d6 (0x6852 bytes) 2024-05-08T08:25:12.901 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:12.901 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:12.902 INFO:teuthology.orchestra.run.smithi031.stdout:422 read 0x71dbaf2 thru 0x71de8d9 (0x2de8 bytes) 2024-05-08T08:25:12.911 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:12.912 INFO:teuthology.orchestra.run.smithi031.stdout:424 writesame 0xe2e7a52 thru 0xe2ea77e data_size 0xf(0x2d2d bytes) 2024-05-08T08:25:12.912 INFO:teuthology.orchestra.run.smithi031.stdout:425 punch from 0xba1056 to 0xbaac8c, (0x9c36 bytes) 2024-05-08T08:25:12.913 INFO:teuthology.orchestra.run.smithi031.stdout:427 write 0x9c967e9 thru 0x9ca060f (0x9e27 bytes) 2024-05-08T08:25:12.917 INFO:teuthology.orchestra.run.smithi031.stdout:428 write 0x862b32e thru 0x8630629 (0x52fc bytes) 2024-05-08T08:25:12.922 INFO:teuthology.orchestra.run.smithi031.stdout:429 read 0xb8e6ef3 thru 0xb8edf15 (0x7023 bytes) 2024-05-08T08:25:12.922 INFO:teuthology.orchestra.run.smithi031.stdout:430 read 0x121b94b thru 0x1222012 (0x66c8 bytes) 2024-05-08T08:25:12.922 INFO:teuthology.orchestra.run.smithi031.stdout:431 write 0x5fbf21f thru 0x5fca9bc (0xb79e bytes) 2024-05-08T08:25:12.927 INFO:teuthology.orchestra.run.smithi031.stdout:434 read 0x1bbf73b thru 0x1bcc7fc (0xd0c2 bytes) 2024-05-08T08:25:12.928 INFO:teuthology.orchestra.run.smithi031.stdout:435 write 0xbf20802 thru 0xbf2f1f8 (0xe9f7 bytes) 2024-05-08T08:25:12.935 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:12.935 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:12.935 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:12.935 INFO:teuthology.orchestra.run.smithi031.stdout:440 punch from 0x3a56d85 to 0x3a631e4, (0xc45f bytes) 2024-05-08T08:25:12.940 INFO:teuthology.orchestra.run.smithi031.stdout:442 writesame 0xec00aa5 thru 0xec01094 data_size 0x14(0x5f0 bytes) 2024-05-08T08:25:12.941 INFO:teuthology.orchestra.run.smithi031.stdout:444 write 0x29616b8 thru 0x2961794 (0xdd bytes) 2024-05-08T08:25:12.945 INFO:teuthology.orchestra.run.smithi031.stdout:445 read 0x35e1df4 thru 0x35ecc93 (0xaea0 bytes) 2024-05-08T08:25:12.945 INFO:teuthology.orchestra.run.smithi031.stdout:446 read 0x112e4a6 thru 0x1135b1f (0x767a bytes) 2024-05-08T08:25:12.952 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:12.960+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:12.959 INFO:teuthology.orchestra.run.smithi031.stdout:447 trunc from 0xec01095 to 0x26861dc 2024-05-08T08:25:12.987 INFO:teuthology.orchestra.run.smithi031.stdout:448 read 0x161e1a thru 0x16f2db (0xd4c2 bytes) 2024-05-08T08:25:12.988 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:12.997 INFO:teuthology.orchestra.run.smithi031.stdout:451 writesame 0x60ab941 thru 0x60b9856 data_size 0x5(0xdf16 bytes) 2024-05-08T08:25:13.007 INFO:teuthology.orchestra.run.smithi031.stdout:452 write 0x9bcf30e thru 0x9bd2d92 (0x3a85 bytes) 2024-05-08T08:25:13.011 INFO:teuthology.orchestra.run.smithi031.stdout:453 read 0x2d29b thru 0x3bc9e (0xea04 bytes) 2024-05-08T08:25:13.012 INFO:teuthology.orchestra.run.smithi031.stdout:454 write 0x1df863b thru 0x1e022f1 (0x9cb7 bytes) 2024-05-08T08:25:13.017 INFO:teuthology.orchestra.run.smithi031.stdout:455 write 0x255ca5b thru 0x255e20a (0x17b0 bytes) 2024-05-08T08:25:13.021 INFO:teuthology.orchestra.run.smithi031.stdout:456 writesame 0x8cb3ba1 thru 0x8cb5249 data_size 0x1(0x16a9 bytes) 2024-05-08T08:25:13.027 INFO:teuthology.orchestra.run.smithi031.stdout:458 trunc from 0x9bd2d93 to 0x837b4b5 2024-05-08T08:25:13.033 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:13.041+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:13.038 INFO:teuthology.orchestra.run.smithi031.stdout:459 read 0x42e75ae thru 0x42f2a61 (0xb4b4 bytes) 2024-05-08T08:25:13.038 INFO:teuthology.orchestra.run.smithi031.stdout:460 read 0x7631faf thru 0x763a72f (0x8781 bytes) 2024-05-08T08:25:13.038 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:13.038 INFO:teuthology.orchestra.run.smithi031.stdout:463 read 0x505b7b6 thru 0x5066472 (0xacbd bytes) 2024-05-08T08:25:13.038 INFO:teuthology.orchestra.run.smithi031.stdout:464 clone 4 order 23 su 524288 sc 2 2024-05-08T08:25:13.067 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:13.067 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:13.076 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:13.160 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:13.168+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:13.222 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:13.223 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:13.231 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:13.235 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:13.242+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:13.336 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:13.343+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:13.377 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:13.377 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:13.386 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:13.437 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:13.444+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:13.489 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T08:25:13.489 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:25:13.489 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T08:25:13.489 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 1.000000 2024-05-08T08:25:13.489 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-05-08T08:25:13.492 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:13.493 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:13.502 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:13.543 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:13.551+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:13.639 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:13.646+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:13.651 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:13.652 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:13.661 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:13.712 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone3 from 0x837b4b5 (overlap 0x26861dc) to 0x0 2024-05-08T08:25:13.739 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:13.746+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:13.795 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:13.795 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:13.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:13.851 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:13.859+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:13.941 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:13.948+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:13.944 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-05-08T08:25:13.950 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:13.950 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:13.954 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-05-08T08:25:13.977 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:14.060 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:14.068+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:14.134 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:14.143 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:14.144 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:14.150+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:14.199 INFO:teuthology.orchestra.run.smithi031.stdout:466 read 0x7a495c2 thru 0x7a4f500 (0x5f3f bytes) 2024-05-08T08:25:14.199 INFO:teuthology.orchestra.run.smithi031.stdout:467 writesame 0x263b304 thru 0x2642e18 data_size 0x1b(0x7b15 bytes) 2024-05-08T08:25:14.224 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.224 INFO:teuthology.orchestra.run.smithi031.stdout:469 read 0x1a58b0a thru 0x1a672b7 (0xe7ae bytes) 2024-05-08T08:25:14.234 INFO:teuthology.orchestra.run.smithi031.stdout:470 writesame 0xac593e8 thru 0xac5a927 data_size 0x28(0x1540 bytes) 2024-05-08T08:25:14.234 INFO:teuthology.orchestra.run.smithi031.stdout:472 write 0x535160c thru 0x5361562 (0xff57 bytes) 2024-05-08T08:25:14.238 INFO:teuthology.orchestra.run.smithi031.stdout:473 write 0x5adc688 thru 0x5ae2154 (0x5acd bytes) 2024-05-08T08:25:14.241 INFO:teuthology.orchestra.run.smithi031.stdout:474 read 0x4fbddc5 thru 0x4fbf973 (0x1baf bytes) 2024-05-08T08:25:14.241 INFO:teuthology.orchestra.run.smithi031.stdout:475 punch from 0x59eca93 to 0x59f93c9, (0xc936 bytes) 2024-05-08T08:25:14.241 INFO:teuthology.orchestra.run.smithi031.stdout:478 write 0x1795e4d thru 0x17a4b14 (0xecc8 bytes) 2024-05-08T08:25:14.243 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:14.251+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:14.282 INFO:teuthology.orchestra.run.smithi031.stdout:480 write 0x8277de0 thru 0x827c4d3 (0x46f4 bytes) 2024-05-08T08:25:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:483 punch from 0x21da314 to 0x21de69e, (0x438a bytes) 2024-05-08T08:25:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.286 INFO:teuthology.orchestra.run.smithi031.stdout:486 write 0xa675f63 thru 0xa67f990 (0x9a2e bytes) 2024-05-08T08:25:14.289 INFO:teuthology.orchestra.run.smithi031.stdout:487 write 0x74916b thru 0x74e98b (0x5821 bytes) 2024-05-08T08:25:14.319 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:14.319 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:14.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:14.337 INFO:teuthology.orchestra.run.smithi031.stdout:490 write 0x12d84fe thru 0x12dccc8 (0x47cb bytes) 2024-05-08T08:25:14.344 INFO:teuthology.orchestra.run.smithi031.stdout:492 trunc from 0xac5a928 to 0x605192a 2024-05-08T08:25:14.344 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:14.352+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:14.373 INFO:teuthology.orchestra.run.smithi031.stdout:494 write 0xdd65fb2 thru 0xdd65ff2 (0x41 bytes) 2024-05-08T08:25:14.378 INFO:teuthology.orchestra.run.smithi031.stdout:495 trunc from 0xdd65ff3 to 0xac5689c 2024-05-08T08:25:14.394 INFO:teuthology.orchestra.run.smithi031.stdout:497 trunc from 0xac5689c to 0x6da5f9b 2024-05-08T08:25:14.401 INFO:teuthology.orchestra.run.smithi031.stdout:499 punch from 0x56248b2 to 0x5625aeb, (0x1239 bytes) 2024-05-08T08:25:14.409 INFO:teuthology.orchestra.run.smithi031.stdout:500 write 0x89bef90 thru 0x89cafb6 (0xc027 bytes) 2024-05-08T08:25:14.413 INFO:teuthology.orchestra.run.smithi031.stdout:503 trunc from 0x89cafb7 to 0x9578c14 2024-05-08T08:25:14.420 INFO:teuthology.orchestra.run.smithi031.stdout:505 write 0x5e1e276 thru 0x5e2856f (0xa2fa bytes) 2024-05-08T08:25:14.422 INFO:teuthology.orchestra.run.smithi031.stdout:507 writesame 0x38ed87d thru 0x38fcc9b data_size 0x1d(0xf41f bytes) 2024-05-08T08:25:14.430 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:14.433 INFO:teuthology.orchestra.run.smithi031.stdout:508 writesame 0xa37f0ec thru 0xa38bbd9 data_size 0x19(0xcaee bytes) 2024-05-08T08:25:14.433 INFO:teuthology.orchestra.run.smithi031.stdout:510 read 0x188ac29 thru 0x1894a49 (0x9e21 bytes) 2024-05-08T08:25:14.434 INFO:teuthology.orchestra.run.smithi031.stdout:511 write 0x573c71a thru 0x5740154 (0x3a3b bytes) 2024-05-08T08:25:14.437 INFO:teuthology.orchestra.run.smithi031.stdout:512 read 0x5e92641 thru 0x5e9c86f (0xa22f bytes) 2024-05-08T08:25:14.437 INFO:teuthology.orchestra.run.smithi031.stdout:513 read 0x2519a4c thru 0x251f7f3 (0x5da8 bytes) 2024-05-08T08:25:14.437 INFO:teuthology.orchestra.run.smithi031.stdout:514 read 0x6bb1406 thru 0x6bb97fc (0x83f7 bytes) 2024-05-08T08:25:14.437 INFO:teuthology.orchestra.run.smithi031.stdout:515 write 0x8c7a0f3 thru 0x8c83058 (0x8f66 bytes) 2024-05-08T08:25:14.442 INFO:teuthology.orchestra.run.smithi031.stdout:516 trunc from 0xa38bbda to 0x909c14e 2024-05-08T08:25:14.445 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:14.452+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:14.449 INFO:teuthology.orchestra.run.smithi031.stdout:517 write 0x1308077 thru 0x13160f8 (0xe082 bytes) 2024-05-08T08:25:14.472 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:14.472 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:14.484 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:14.501 INFO:teuthology.orchestra.run.smithi031.stdout:518 punch from 0x8cbcbbe to 0x8cc373a, (0x6b7c bytes) 2024-05-08T08:25:14.501 INFO:teuthology.orchestra.run.smithi031.stdout:519 read 0x401e7f2 thru 0x40252d5 (0x6ae4 bytes) 2024-05-08T08:25:14.501 INFO:teuthology.orchestra.run.smithi031.stdout:520 read 0x118948a thru 0x1196457 (0xcfce bytes) 2024-05-08T08:25:14.501 INFO:teuthology.orchestra.run.smithi031.stdout:523 read 0x52de368 thru 0x52e08da (0x2573 bytes) 2024-05-08T08:25:14.501 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.501 INFO:teuthology.orchestra.run.smithi031.stdout:525 punch from 0x4834a3d to 0x4842d32, (0xe2f5 bytes) 2024-05-08T08:25:14.514 INFO:teuthology.orchestra.run.smithi031.stdout:527 write 0xc4b2edf thru 0xc4c20f4 (0xf216 bytes) 2024-05-08T08:25:14.523 INFO:teuthology.orchestra.run.smithi031.stdout:528 write 0xe7cb894 thru 0xe7d8a0e (0xd17b bytes) 2024-05-08T08:25:14.526 INFO:teuthology.orchestra.run.smithi031.stdout:529 read 0xc30d19 thru 0xc3a55c (0x9844 bytes) 2024-05-08T08:25:14.527 INFO:teuthology.orchestra.run.smithi031.stdout:530 write 0xdd9edc6 thru 0xdda086a (0x1aa5 bytes) 2024-05-08T08:25:14.530 INFO:teuthology.orchestra.run.smithi031.stdout:533 write 0xe45af99 thru 0xe45fcfc (0x4d64 bytes) 2024-05-08T08:25:14.533 INFO:teuthology.orchestra.run.smithi031.stdout:534 read 0xb92c55b thru 0xb93521c (0x8cc2 bytes) 2024-05-08T08:25:14.533 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.533 INFO:teuthology.orchestra.run.smithi031.stdout:536 write 0x879c1f thru 0x88807a (0xe45c bytes) 2024-05-08T08:25:14.546 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:14.554+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:14.586 INFO:teuthology.orchestra.run.smithi031.stdout:537 trunc from 0xe7d8a0f to 0x7077add 2024-05-08T08:25:14.592 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:14.592 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:14.612 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:14.612 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:14.613 INFO:teuthology.orchestra.run.smithi031.stdout:538 write 0xdb2fc93 thru 0xdb32a2c (0x2d9a bytes) 2024-05-08T08:25:14.617 INFO:teuthology.orchestra.run.smithi031.stdout:539 write 0xaa405a5 thru 0xaa49609 (0x9065 bytes) 2024-05-08T08:25:14.619 INFO:teuthology.orchestra.run.smithi031.stdout:540 writesame 0x750cf1e thru 0x750e1a4 data_size 0x1f(0x1287 bytes) 2024-05-08T08:25:14.622 INFO:teuthology.orchestra.run.smithi031.stdout:541 punch from 0x1db7742 to 0x1dba857, (0x3115 bytes) 2024-05-08T08:25:14.623 INFO:teuthology.orchestra.run.smithi031.stdout:542 read 0xbed592d thru 0xbed69c9 (0x109d bytes) 2024-05-08T08:25:14.623 INFO:teuthology.orchestra.run.smithi031.stdout:543 write 0x4a24459 thru 0x4a328cf (0xe477 bytes) 2024-05-08T08:25:14.627 INFO:teuthology.orchestra.run.smithi031.stdout:544 writesame 0x178e436 thru 0x17919e7 data_size 0x3a(0x35b2 bytes) 2024-05-08T08:25:14.628 INFO:teuthology.orchestra.run.smithi031.stdout:545 read 0x556477c thru 0x556ec38 (0xa4bd bytes) 2024-05-08T08:25:14.629 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.632 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:14.632 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:14.639 INFO:teuthology.orchestra.run.smithi031.stdout:548 trunc from 0xdb32a2d to 0x5c159d3 2024-05-08T08:25:14.642 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:14.642 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:14.642 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:14.647 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:14.654+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:14.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:25:14.654 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:14.654 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:14.655 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:14.660 INFO:teuthology.orchestra.run.smithi031.stdout:550 trunc from 0x5c159d3 to 0xd756ad4 2024-05-08T08:25:14.667 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.667 INFO:teuthology.orchestra.run.smithi031.stdout:552 write 0x6cf112c thru 0x6cfda55 (0xc92a bytes) 2024-05-08T08:25:14.676 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:14.677 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:14.678 INFO:teuthology.orchestra.run.smithi031.stdout:553 write 0xea22c74 thru 0xea2a83a (0x7bc7 bytes) 2024-05-08T08:25:14.682 INFO:teuthology.orchestra.run.smithi031.stdout:554 write 0x70f292e thru 0x70f5bee (0x32c1 bytes) 2024-05-08T08:25:14.685 INFO:teuthology.orchestra.run.smithi031.stdout:555 writesame 0xbebc519 thru 0xbecb0f8 data_size 0x22(0xebe0 bytes) 2024-05-08T08:25:14.690 INFO:teuthology.orchestra.run.smithi031.stdout:556 writesame 0xac98e7c thru 0xaca2d5d data_size 0x3(0x9ee2 bytes) 2024-05-08T08:25:14.698 INFO:teuthology.orchestra.run.smithi031.stdout:559 writesame 0xb631294 thru 0xb640bc8 data_size 0x1(0xf935 bytes) 2024-05-08T08:25:14.699 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:14.699 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:14.722 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:14.723 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:14.733 INFO:teuthology.orchestra.run.smithi031.stdout:562 write 0xb1f8569 thru 0xb201662 (0x90fa bytes) 2024-05-08T08:25:14.735 INFO:teuthology.orchestra.run.smithi031.stdout:563 write 0x3ad3d89 thru 0x3ad9d9a (0x6012 bytes) 2024-05-08T08:25:14.738 INFO:teuthology.orchestra.run.smithi031.stdout:564 trunc from 0xea2a83b to 0xddc7d72 2024-05-08T08:25:14.747 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:14.755+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:14.748 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:14.748 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:14.751 INFO:teuthology.orchestra.run.smithi031.stdout:565 punch from 0xc0e4f51 to 0xc0ef3f3, (0xa4a2 bytes) 2024-05-08T08:25:14.751 INFO:teuthology.orchestra.run.smithi031.stdout:566 writesame 0x7fefec3 thru 0x7ff4d39 data_size 0x1(0x4e77 bytes) 2024-05-08T08:25:14.761 INFO:teuthology.orchestra.run.smithi031.stdout:567 read 0xa8e7e47 thru 0xa8ef330 (0x74ea bytes) 2024-05-08T08:25:14.761 INFO:teuthology.orchestra.run.smithi031.stdout:571 read 0x21aca5c thru 0x21b693d (0x9ee2 bytes) 2024-05-08T08:25:14.762 INFO:teuthology.orchestra.run.smithi031.stdout:574 write 0xc4838c3 thru 0xc491623 (0xdd61 bytes) 2024-05-08T08:25:14.766 INFO:teuthology.orchestra.run.smithi031.stdout:575 read 0xa614b4e thru 0xa6192cc (0x477f bytes) 2024-05-08T08:25:14.766 INFO:teuthology.orchestra.run.smithi031.stdout:576 read 0x9f69726 thru 0x9f6c995 (0x3270 bytes) 2024-05-08T08:25:14.766 INFO:teuthology.orchestra.run.smithi031.stdout:577 punch from 0xb9329f8 to 0xb9418b8, (0xeec0 bytes) 2024-05-08T08:25:14.766 INFO:teuthology.orchestra.run.smithi031.stdout:578 writesame 0x11beb38 thru 0x11c8f49 data_size 0x1(0xa412 bytes) 2024-05-08T08:25:14.782 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:1 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'1, mlcod=0'0)", 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.167637276000001, 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0075841290000000002, 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.783 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621504+0000", 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621510+0000", 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9610000000000002e-06 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621517+0000", 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0949999999999998e-06 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621526+0000", 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1919999999999994e-06 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621572+0000", 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5599000000000002e-05 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.621600+0000", 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8399e-05 2024-05-08T08:25:14.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.629024+0000", 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074240119999999998 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.629088+0000", 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3455999999999997e-05 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:83 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.067452466999999, 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052916229999999996, 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721688+0000", 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721689+0000", 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5499999999999998e-07 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721697+0000", 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2449999999999998e-06 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721706+0000", 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0000000000000002e-06 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721713+0000", 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2380000000000002e-06 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721822+0000", 2024-05-08T08:25:14.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010911 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.721845+0000", 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3833999999999999e-05 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726930+0000", 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050846830000000004 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726980+0000", 2024-05-08T08:25:14.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9957999999999998e-05 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:84 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.066985032000002, 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048602560000000003, 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722156+0000", 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722157+0000", 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7899999999999998e-07 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722191+0000", 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4762999999999999e-05 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722201+0000", 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2380000000000003e-06 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722209+0000", 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8019999999999998e-06 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722251+0000", 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1128999999999999e-05 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.722268+0000", 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7371e-05 2024-05-08T08:25:14.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.726987+0000", 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047187699999999997 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.727016+0000", 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9504000000000001e-05 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.868487837, 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073213000000000002, 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920653+0000", 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920654+0000", 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8800000000000003e-07 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920686+0000", 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2338000000000001e-05 2024-05-08T08:25:14.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920695+0000", 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5650000000000007e-06 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920701+0000", 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8170000000000004e-06 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920749+0000", 2024-05-08T08:25:14.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8071000000000001e-05 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.920768+0000", 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8546000000000002e-05 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927915+0000", 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0071468269999999997 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927974+0000", 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9648000000000002e-05 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:107 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'69, mlcod=16'64)", 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.867961215000001, 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0068250300000000002, 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.801 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921180+0000", 2024-05-08T08:25:14.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5199999999999997e-07 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921185+0000", 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.135e-06 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921195+0000", 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.533e-06 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921201+0000", 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5069999999999997e-06 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921321+0000", 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011977 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921339+0000", 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7608000000000001e-05 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.927977+0000", 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0066382029999999996 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928005+0000", 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7722e-05 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:108 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'70, mlcod=16'64)", 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.867893688999999, 2024-05-08T08:25:14.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067827720000000003, 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921247+0000", 2024-05-08T08:25:14.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921248+0000", 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921252+0000", 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4449999999999998e-06 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921261+0000", 2024-05-08T08:25:14.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5830000000000007e-06 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921267+0000", 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8459999999999996e-06 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921577+0000", 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031014399999999999 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921595+0000", 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7802e-05 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928006+0000", 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0064114100000000002 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928030+0000", 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4042000000000001e-05 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:109 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'64)", 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.867315048000002, 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0062286720000000002, 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921826+0000", 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.44e-07 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921855+0000", 2024-05-08T08:25:14.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8645000000000001e-05 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921865+0000", 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7310000000000005e-06 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921871+0000", 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.263e-06 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921924+0000", 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3109999999999998e-05 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.921945+0000", 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1118e-05 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928031+0000", 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0060861200000000004 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928055+0000", 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3141000000000001e-05 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:110 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'72, mlcod=16'64)", 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.866843011, 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057807739999999998, 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922298+0000", 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1699999999999999e-07 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922304+0000", 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.288e-05 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922323+0000", 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3749999999999999e-06 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922387+0000", 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3932999999999999e-05 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922407+0000", 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9885e-05 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928056+0000", 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056492180000000001 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928079+0000", 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2467e-05 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.818 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:111 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'73, mlcod=16'64)", 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.866497196000001, 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005461502, 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922644+0000", 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2e-07 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922649+0000", 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1200000000000001e-06 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922658+0000", 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8419999999999994e-06 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922664+0000", 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1290000000000001e-06 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922725+0000", 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922743+0000", 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8142999999999999e-05 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928080+0000", 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053372430000000002 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928105+0000", 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5004999999999999e-05 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'64)", 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.866179884000001, 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055842340000000004, 2024-05-08T08:25:14.823 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922961+0000", 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.922962+0000", 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923087+0000", 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012524199999999999 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923097+0000", 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8700000000000004e-06 2024-05-08T08:25:14.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923103+0000", 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2049999999999999e-06 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923163+0000", 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0554000000000003e-05 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923182+0000", 2024-05-08T08:25:14.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8515000000000001e-05 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928508+0000", 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053261330000000003 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928545+0000", 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7165000000000001e-05 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4207.0:113 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'75, mlcod=16'64)", 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 16.865602092, 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050342659999999999, 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923539+0000", 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0200000000000002e-07 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923547+0000", 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2119999999999997e-06 2024-05-08T08:25:14.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923556+0000", 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3470000000000005e-06 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923564+0000", 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8779999999999998e-06 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923625+0000", 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1309999999999994e-05 2024-05-08T08:25:14.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.923644+0000", 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8403999999999999e-05 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928547+0000", 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0049035349999999997 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:24:57.928573+0000", 2024-05-08T08:25:14.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5978000000000001e-05 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1825 3.4 3:23b8ebb3:::rbd_data.119116cc7dd0.00000000000001e0:head [set-alloc-hint object_size 262144 write_size 262144,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.251364+0000", 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.5377765620000003, 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004867405, 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:14.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1825 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251364+0000", 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251364+0000", 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251366+0000", 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3820000000000001e-06 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251378+0000", 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1841999999999999e-05 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251394+0000", 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6398000000000001e-05 2024-05-08T08:25:14.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251402+0000", 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4179999999999997e-06 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251457+0000", 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4249000000000002e-05 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251504+0000", 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7549000000000002e-05 2024-05-08T08:25:14.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.251582+0000", 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7501000000000002e-05 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.252660+0000", 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010780970000000001 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256143+0000", 2024-05-08T08:25:14.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003483089 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256184+0000", 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1291000000000002e-05 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256232+0000", 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7589000000000001e-05 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2371 3.8 e24/18 3:158ebef6:::rbd_data.119116cc7dd0.0000000000000204:head v 24'42, mlcod=24'39)", 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721203+0000", 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0679378009999998, 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0086481509999999998, 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721203+0000", 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721203+0000", 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721204+0000", 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6199999999999997e-07 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721211+0000", 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6460000000000004e-06 2024-05-08T08:25:14.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721221+0000", 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.045e-05 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721227+0000", 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2179999999999996e-06 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721298+0000", 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0449000000000003e-05 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721320+0000", 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2795000000000001e-05 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729727+0000", 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084063660000000002 2024-05-08T08:25:14.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729851+0000", 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012446500000000001 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2374 3.1a e24/18 3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head v 24'15, mlcod=23'12)", 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.722213+0000", 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0669279239999998, 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010496847, 2024-05-08T08:25:14.842 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722213+0000", 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722213+0000", 2024-05-08T08:25:14.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722213+0000", 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722219+0000", 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6840000000000001e-06 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722229+0000", 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8109999999999997e-06 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722236+0000", 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1600000000000001e-06 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722302+0000", 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6192999999999995e-05 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722323+0000", 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0988999999999998e-05 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732650+0000", 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010326465999999999 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732710+0000", 2024-05-08T08:25:14.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0154e-05 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2377 3.8 e24/18 3:17a99b47:::rbd_data.119116cc7dd0.000000000000038a:head v 24'44, mlcod=24'39)", 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.722833+0000", 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0663079030000002, 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098698659999999997, 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.847 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722833+0000", 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722833+0000", 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722834+0000", 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8899999999999998e-07 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722841+0000", 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8469999999999998e-06 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722851+0000", 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0706999999999999e-05 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722859+0000", 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0319999999999998e-06 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722950+0000", 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1458000000000004e-05 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722973+0000", 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3281e-05 2024-05-08T08:25:14.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732654+0000", 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096805529999999997 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732703+0000", 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9098999999999999e-05 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.851 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2381 3.0 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725070+0000", 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.0640711409999999, 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080516900000000002, 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2381 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725070+0000", 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725070+0000", 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725071+0000", 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6029999999999999e-06 2024-05-08T08:25:14.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725077+0000", 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725091+0000", 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4081e-05 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725099+0000", 2024-05-08T08:25:14.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1159999999999999e-06 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725161+0000", 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2392999999999997e-05 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725256+0000", 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4524999999999994e-05 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:14.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725423+0000", 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016687200000000001 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.730286+0000", 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048634589999999997 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.733012+0000", 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027257269999999998 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.733071+0000", 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9138999999999997e-05 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.733121+0000", 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0343999999999998e-05 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2640 3.4 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 7226325~27732] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:14.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.993364+0000", 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7957774280000001, 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0065402450000000001, 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2640 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.993364+0000", 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.993364+0000", 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.993365+0000", 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.669e-06 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.994302+0000", 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093638200000000003 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.994323+0000", 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1495999999999999e-05 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.994331+0000", 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4190000000000002e-06 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.994406+0000", 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4167999999999999e-05 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.994585+0000", 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000179748 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995031+0000", 2024-05-08T08:25:14.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00044597199999999999 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999636+0000", 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046047229999999998 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999805+0000", 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016861099999999999 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999844+0000", 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8911000000000001e-05 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999904+0000", 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0146000000000003e-05 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2784 3.4 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:12.068190+0000", 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7209509179999998, 2024-05-08T08:25:14.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016254596, 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2784 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068190+0000", 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068190+0000", 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068191+0000", 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8299999999999997e-07 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068195+0000", 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7600000000000002e-06 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068203+0000", 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3769999999999998e-06 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068210+0000", 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5750000000000003e-06 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068263+0000", 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2305000000000002e-05 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.068345+0000", 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2517999999999998e-05 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.070960+0000", 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026150449999999999 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.084274+0000", 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013314196 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.084327+0000", 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2407000000000002e-05 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.084390+0000", 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3212000000000004e-05 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.084445+0000", 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4718e-05 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4497.0:2784 3.4 e24/18 ondisk, result = 0)", 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:12.078805+0000", 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.7103358819999999, 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005649304, 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.078805+0000", 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.078805+0000", 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.078806+0000", 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6700000000000002e-07 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.078809+0000", 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9380000000000001e-06 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.078820+0000", 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.131e-05 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.078826+0000", 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000002e-06 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.084308+0000", 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054820069999999997 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.084319+0000", 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0919e-05 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.084454+0000", 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000134969 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6441 3.0 3:0362a1e0:::rbd_data.119152d8614e.000000000000000f:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.759489+0000", 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.029652418, 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0097468239999999994, 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:14.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6441 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759489+0000", 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759489+0000", 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759491+0000", 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1950000000000002e-06 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759500+0000", 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2029999999999998e-06 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759518+0000", 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8535000000000001e-05 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759528+0000", 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7699999999999996e-06 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759587+0000", 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8690000000000002e-05 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.759640+0000", 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3065000000000001e-05 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.761268+0000", 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001627905 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.769059+0000", 2024-05-08T08:25:14.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0077911539999999998 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.769116+0000", 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7331999999999999e-05 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.769182+0000", 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5279999999999998e-05 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:14.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.769235+0000", 2024-05-08T08:25:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3695e-05 2024-05-08T08:25:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:14.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:14.879 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:14.880 INFO:teuthology.orchestra.run.smithi031.stdout:579 read 0x6b985e4 thru 0x6b9b7ce (0x31eb bytes) 2024-05-08T08:25:14.880 INFO:teuthology.orchestra.run.smithi031.stdout:580 punch from 0x42bf937 to 0x42c9896, (0x9f5f bytes) 2024-05-08T08:25:14.880 INFO:teuthology.orchestra.run.smithi031.stdout:581 punch from 0x931ca8d to 0x9321bb7, (0x512a bytes) 2024-05-08T08:25:14.880 INFO:teuthology.orchestra.run.smithi031.stdout:582 read 0x913af1f thru 0x9144bb8 (0x9c9a bytes) 2024-05-08T08:25:14.880 INFO:teuthology.orchestra.run.smithi031.stdout:583 write 0x574e480 thru 0x575dca8 (0xf829 bytes) 2024-05-08T08:25:14.881 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:584 trunc from 0xddc7d72 to 0xa37e0b1 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:585 punch from 0x24e44cb to 0x24e81b3, (0x3ce8 bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:588 read 0xe9675f thru 0xe97381 (0xc23 bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:589 read 0x10e5430 thru 0x10eb973 (0x6544 bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:592 write 0xe7b6fec thru 0xe7c489f (0xd8b4 bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:593 write 0xd9e493f thru 0xd9f0bd5 (0xc297 bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:594 read 0x804804d thru 0x804c7da (0x478e bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:596 write 0x76e77fe thru 0x76ea3ef (0x2bf2 bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:597 write 0x37500ec thru 0x37538b9 (0x37ce bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:598 trunc from 0xe7c48a0 to 0x3f57915 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:599 write 0x3e3b03a thru 0x3e3f6e7 (0x46ae bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:600 punch from 0x3dc2802 to 0x3dca439, (0x7c37 bytes) 2024-05-08T08:25:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:603 writesame 0xa4f1e87 thru 0xa4fd3ef data_size 0x1(0xb569 bytes) 2024-05-08T08:25:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:604 punch from 0x1b77ac7 to 0x1b78449, (0x982 bytes) 2024-05-08T08:25:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:605 writesame 0xc0b69b5 thru 0xc0bb1e2 data_size 0x1(0x482e bytes) 2024-05-08T08:25:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:606 punch from 0x27ecd9c to 0x27ed79c, (0xa00 bytes) 2024-05-08T08:25:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:607 read 0x8c1eb54 thru 0x8c2b26e (0xc71b bytes) 2024-05-08T08:25:14.885 INFO:teuthology.orchestra.run.smithi031.stdout:609 write 0xc1423ca thru 0xc1462d6 (0x3f0d bytes) 2024-05-08T08:25:14.885 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:14.887+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:14.886 INFO:teuthology.orchestra.run.smithi031.stdout:610 trunc from 0xc1462d7 to 0xa9767d1 2024-05-08T08:25:14.895 INFO:teuthology.orchestra.run.smithi031.stdout:613 trunc from 0xa9767d1 to 0xdb0db11 2024-05-08T08:25:14.900 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.901 INFO:teuthology.orchestra.run.smithi031.stdout:615 read 0xa30d51 thru 0xa3954c (0x87fc bytes) 2024-05-08T08:25:14.901 INFO:teuthology.orchestra.run.smithi031.stdout:616 writesame 0x1ebe77 thru 0x1f4480 data_size 0x2a(0x860a bytes) 2024-05-08T08:25:14.903 INFO:teuthology.orchestra.run.smithi031.stdout:617 writesame 0x8c18f86 thru 0x8c20818 data_size 0x1(0x7893 bytes) 2024-05-08T08:25:14.920 INFO:teuthology.orchestra.run.smithi031.stdout:619 write 0x3636a0c thru 0x363e75f (0x7d54 bytes) 2024-05-08T08:25:14.921 INFO:teuthology.orchestra.run.smithi031.stdout:622 read 0x7e0e02c thru 0x7e1d87e (0xf853 bytes) 2024-05-08T08:25:14.921 INFO:teuthology.orchestra.run.smithi031.stdout:625 read 0x962eac thru 0x9728d9 (0xfa2e bytes) 2024-05-08T08:25:14.923 INFO:teuthology.orchestra.run.smithi031.stdout:626 read 0xc6e3676 thru 0xc6f07e6 (0xd171 bytes) 2024-05-08T08:25:14.923 INFO:teuthology.orchestra.run.smithi031.stdout:627 read 0xcbe67af thru 0xcbef86e (0x90c0 bytes) 2024-05-08T08:25:14.923 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.923 INFO:teuthology.orchestra.run.smithi031.stdout:629 writesame 0xfec4f5 thru 0xffc105 data_size 0x1(0xfc11 bytes) 2024-05-08T08:25:14.947 INFO:teuthology.orchestra.run.smithi031.stdout:631 writesame 0x86b715d thru 0x86b8bbf data_size 0x1(0x1a63 bytes) 2024-05-08T08:25:14.952 INFO:teuthology.orchestra.run.smithi031.stdout:632 read 0x67c6008 thru 0x67ca1c1 (0x41ba bytes) 2024-05-08T08:25:14.962 INFO:teuthology.orchestra.run.smithi031.stdout:633 trunc from 0xdb0db11 to 0x5617309 2024-05-08T08:25:14.971 INFO:teuthology.orchestra.run.smithi031.stdout:636 read 0x25411a4 thru 0x2545276 (0x40d3 bytes) 2024-05-08T08:25:14.972 INFO:teuthology.orchestra.run.smithi031.stdout:638 punch from 0x1ee364 to 0x1fd280, (0xef1c bytes) 2024-05-08T08:25:14.981 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:14.989+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:14.982 INFO:teuthology.orchestra.run.smithi031.stdout:639 trunc from 0x5617309 to 0xe2dcb2b 2024-05-08T08:25:14.988 INFO:teuthology.orchestra.run.smithi031.stdout:640 write 0x862b4f6 thru 0x8637325 (0xbe30 bytes) 2024-05-08T08:25:14.991 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.991 INFO:teuthology.orchestra.run.smithi031.stdout:642 write 0x57495d3 thru 0x575658e (0xcfbc bytes) 2024-05-08T08:25:14.995 INFO:teuthology.orchestra.run.smithi031.stdout:643 read 0xa55d578 thru 0xa568fb9 (0xba42 bytes) 2024-05-08T08:25:14.995 INFO:teuthology.orchestra.run.smithi031.stdout:645 read 0xe2d3ba5 thru 0xe2dcb2a (0x8f86 bytes) 2024-05-08T08:25:14.995 INFO:teuthology.orchestra.run.smithi031.stdout:646 read 0x5c1b9a2 thru 0x5c23c93 (0x82f2 bytes) 2024-05-08T08:25:14.995 INFO:teuthology.orchestra.run.smithi031.stdout:651 writesame 0x4d3e0f0 thru 0x4d404db data_size 0x13(0x23ec bytes) 2024-05-08T08:25:14.998 INFO:teuthology.orchestra.run.smithi031.stdout:652 read 0xc8cf683 thru 0xc8d7895 (0x8213 bytes) 2024-05-08T08:25:14.998 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:14.999 INFO:teuthology.orchestra.run.smithi031.stdout:654 trunc from 0xe2dcb2b to 0xd8de8ca 2024-05-08T08:25:15.012 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:15.012 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:15.012 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:15.019 INFO:teuthology.orchestra.run.smithi031.stdout:656 trunc from 0xd8de8ca to 0x1bcfaef 2024-05-08T08:25:15.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:15.045 INFO:teuthology.orchestra.run.smithi031.stdout:657 write 0x8124b5b thru 0x81251cd (0x673 bytes) 2024-05-08T08:25:15.049 INFO:teuthology.orchestra.run.smithi031.stdout:660 write 0x10e9d03 thru 0x10f3722 (0x9a20 bytes) 2024-05-08T08:25:15.060 INFO:teuthology.orchestra.run.smithi031.stdout:661 write 0xb978c96 thru 0xb97d170 (0x44db bytes) 2024-05-08T08:25:15.063 INFO:teuthology.orchestra.run.smithi031.stdout:662 read 0x9b51e97 thru 0x9b54f0e (0x3078 bytes) 2024-05-08T08:25:15.063 INFO:teuthology.orchestra.run.smithi031.stdout:665 read 0xa8aa7d thru 0xa91736 (0x6cba bytes) 2024-05-08T08:25:15.063 INFO:teuthology.orchestra.run.smithi031.stdout:666 read 0x5537d40 thru 0x553e2d0 (0x6591 bytes) 2024-05-08T08:25:15.063 INFO:teuthology.orchestra.run.smithi031.stdout:667 read 0x5d9941d thru 0x5da782c (0xe410 bytes) 2024-05-08T08:25:15.063 INFO:teuthology.orchestra.run.smithi031.stdout:670 clone 5 order 25 su 262144 sc 10 2024-05-08T08:25:15.081 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:15.089+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:15.156 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:15.156 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:15.156 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:15.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:15.205 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:15.213+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:15.283 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:15.290+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:15.315 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1729 3.9 e23/18 3:9695f221:::rbd_data.119116cc7dd0.00000000000002df:head v 23'70, mlcod=23'68)", 2024-05-08T08:25:15.315 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1499983110000001, 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001912813, 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171947+0000", 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2610000000000001e-06 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171962+0000", 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4827999999999999e-05 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171983+0000", 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1542000000000001e-05 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172004+0000", 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0577e-05 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172072+0000", 2024-05-08T08:25:15.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8205000000000002e-05 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172097+0000", 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5004999999999999e-05 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173793+0000", 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001696513 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173858+0000", 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4882000000000007e-05 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1805 3.4 e23/18 3:247a66e4:::rbd_data.119116cc7dd0.0000000000000088:head v 23'26, mlcod=22'24)", 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0885250879999999, 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002584514, 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233420+0000", 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7100000000000011e-07 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233438+0000", 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8084000000000002e-05 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233454+0000", 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5851000000000001e-05 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233463+0000", 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9819999999999997e-06 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233527+0000", 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4140999999999995e-05 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233551+0000", 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4606000000000001e-05 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.235949+0000", 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002397475 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236003+0000", 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4404000000000003e-05 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1816 3.4 e23/18 3:22bef275:::rbd_data.119116cc7dd0.0000000000000147:head v 23'27, mlcod=23'25)", 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:15.324 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0762175389999999, 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012053820000000001, 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:15.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245727+0000", 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0699999999999999e-06 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245745+0000", 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7932999999999999e-05 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245763+0000", 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8139999999999999e-05 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245772+0000", 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2789999999999998e-06 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245849+0000", 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7167000000000002e-05 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245876+0000", 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7260999999999999e-05 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.246893+0000", 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001017015 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.246932+0000", 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8516999999999997e-05 2024-05-08T08:25:15.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1829 3.4 e23/18 3:243982ca:::rbd_data.119116cc7dd0.00000000000001e4:head v 23'29, mlcod=23'27)", 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.0652325869999997, 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002557368, 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256712+0000", 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1680000000000001e-06 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256722+0000", 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3910000000000004e-06 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256732+0000", 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0682999999999999e-05 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256738+0000", 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-05-08T08:25:15.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256783+0000", 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5015000000000002e-05 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256796+0000", 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3745e-05 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259238+0000", 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024414409999999999 2024-05-08T08:25:15.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259269+0000", 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0648999999999997e-05 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1903 3.4 e23/18 3:24e0ebf3:::rbd_data.119116cc7dd0.0000000000000376:head v 23'30, mlcod=23'28)", 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:15.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.9960689809999996, 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015443570000000001, 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:15.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325876+0000", 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0020000000000001e-06 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325906+0000", 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9688000000000001e-05 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325920+0000", 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4558e-05 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325927+0000", 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5459999999999997e-06 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325980+0000", 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3029999999999999e-05 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325996+0000", 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5988999999999999e-05 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.327370+0000", 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001374632 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.327419+0000", 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8912000000000002e-05 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2088 3.4 e23/18 3:206618ae:::rbd_data.119116cc7dd0.0000000000000195:head v 23'37, mlcod=23'35)", 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8348546280000004, 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001240746, 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:15.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487090+0000", 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.189e-06 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487103+0000", 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2685e-05 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487121+0000", 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8134e-05 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487130+0000", 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.833e-06 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487193+0000", 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2898999999999995e-05 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487210+0000", 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6946e-05 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.488287+0000", 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001077089 2024-05-08T08:25:15.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.488330+0000", 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2970999999999999e-05 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2113 3.4 e23/18 3:23bb875f:::rbd_data.119116cc7dd0.00000000000000a6:head v 23'38, mlcod=23'36)", 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8117612000000003, 2024-05-08T08:25:15.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020550019999999998, 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:15.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510184+0000", 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.387e-06 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510206+0000", 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2084999999999999e-05 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510226+0000", 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0046000000000001e-05 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510238+0000", 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1843e-05 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510297+0000", 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8959999999999998e-05 2024-05-08T08:25:15.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510313+0000", 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5787999999999999e-05 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512193+0000", 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001879999 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512238+0000", 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4894000000000001e-05 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2121 3.4 e23/18 3:27cd253f:::rbd_data.119116cc7dd0.0000000000000156:head v 23'39, mlcod=23'37)", 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.8061865480000003, 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023924910000000001, 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515758+0000", 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999996e-07 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515777+0000", 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8817999999999998e-05 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515792+0000", 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5277999999999998e-05 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515800+0000", 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6000000000000001e-06 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515868+0000", 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7834000000000001e-05 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515899+0000", 2024-05-08T08:25:15.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1473999999999997e-05 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.518104+0000", 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022054000000000002 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.518150+0000", 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5318000000000002e-05 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2137 3.4 e23/18 3:206618ae:::rbd_data.119116cc7dd0.0000000000000195:head v 23'41, mlcod=23'38)", 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7804387239999997, 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027337440000000002, 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541506+0000", 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-07 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541511+0000", 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5550000000000002e-06 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541523+0000", 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.207e-05 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541530+0000", 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541599+0000", 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8143000000000001e-05 2024-05-08T08:25:15.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541617+0000", 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8794999999999999e-05 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544188+0000", 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025705290000000002 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544239+0000", 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0967000000000002e-05 2024-05-08T08:25:15.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2141 3.9 e23/18 3:9250a055:::rbd_data.119116cc7dd0.000000000000027d:head v 23'88, mlcod=23'86)", 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7774630240000002, 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023341719999999998, 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:15.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544482+0000", 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544487+0000", 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1590000000000001e-06 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544499+0000", 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2621e-05 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544506+0000", 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7510000000000002e-06 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544577+0000", 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0801000000000002e-05 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544599+0000", 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1642e-05 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546771+0000", 2024-05-08T08:25:15.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021727180000000001 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546815+0000", 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3683000000000002e-05 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2207 3.9 e23/18 3:961d4b12:::rbd_data.119116cc7dd0.00000000000000cf:head v 23'91, mlcod=23'89)", 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.7205071590000003, 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001369622, 2024-05-08T08:25:15.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601438+0000", 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6840000000000001e-06 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601456+0000", 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7320999999999999e-05 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601475+0000", 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9448000000000001e-05 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601484+0000", 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601554+0000", 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9820000000000006e-05 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601581+0000", 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6517e-05 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.602759+0000", 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001178183 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.602806+0000", 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7311000000000002e-05 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2329 3.0 e24/18 3:071e2661:::rbd_data.119116cc7dd0.0000000000000012:head v 24'42, mlcod=23'39)", 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.708666+0000", 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6132782720000001, 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013530300000000001, 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708666+0000", 2024-05-08T08:25:15.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708666+0000", 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708667+0000", 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2419999999999999e-06 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708688+0000", 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1330000000000001e-05 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708709+0000", 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1030999999999999e-05 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708719+0000", 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8260000000000005e-06 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708783+0000", 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4226999999999993e-05 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708805+0000", 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2065999999999998e-05 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709982+0000", 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011766179999999999 2024-05-08T08:25:15.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.710019+0000", 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6690000000000003e-05 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2337 3.0 e24/18 3:04522806:::rbd_data.119116cc7dd0.000000000000001a:head v 24'44, mlcod=23'39)", 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.709397+0000", 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6125466390000001, 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011970329999999999, 2024-05-08T08:25:15.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709397+0000", 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709397+0000", 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709398+0000", 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.37e-07 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709405+0000", 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4599999999999997e-06 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709415+0000", 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3789999999999995e-06 2024-05-08T08:25:15.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709420+0000", 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3140000000000003e-06 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709475+0000", 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439000000000002e-05 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709499+0000", 2024-05-08T08:25:15.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3425999999999999e-05 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.710558+0000", 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001059143 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.710594+0000", 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6134999999999999e-05 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2351 3.4 e24/18 3:23bb875f:::rbd_data.119116cc7dd0.00000000000000a6:head v 24'44, mlcod=23'41)", 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.714884+0000", 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.6070602740000002, 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001342593, 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714884+0000", 2024-05-08T08:25:15.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714884+0000", 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714892+0000", 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6309999999999992e-06 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714902+0000", 2024-05-08T08:25:15.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714922+0000", 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0098999999999998e-05 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714933+0000", 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1204e-05 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714993+0000", 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9670000000000003e-05 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.715016+0000", 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3224e-05 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.716182+0000", 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001165983 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.716226+0000", 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3785999999999997e-05 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2370 3.4 e24/18 3:237be9c1:::rbd_data.119116cc7dd0.00000000000001eb:head v 24'46, mlcod=23'42)", 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.600657354, 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029578109999999999, 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721287+0000", 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7199999999999997e-07 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721294+0000", 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9800000000000001e-06 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721309+0000", 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5116999999999999e-05 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9030000000000004e-06 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721383+0000", 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6111000000000005e-05 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721406+0000", 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2453e-05 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724192+0000", 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027864909999999999 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724244+0000", 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.5964440469999999, 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:25:15.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.3267505100000001, 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:15.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2784 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'50, mlcod=24'48)", 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:12.075083+0000", 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.2468608080000001, 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011127400000000001, 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075083+0000", 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075083+0000", 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075084+0000", 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0419999999999999e-06 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075105+0000", 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1433e-05 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075121+0000", 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5608999999999999e-05 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075129+0000", 2024-05-08T08:25:15.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6260000000000003e-06 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075191+0000", 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2131000000000005e-05 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.075215+0000", 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.389e-05 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.076156+0000", 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094112500000000001 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.076196+0000", 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9884e-05 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:6406 3.4 e27/18 3:22113792:::rbd_data.119152d8614e.000000000000000d:head v 27'66, mlcod=26'64)", 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.677729+0000", 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.64421448100000001, 2024-05-08T08:25:15.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001173372, 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677729+0000", 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677729+0000", 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677730+0000", 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1370000000000001e-06 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677774+0000", 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3392000000000001e-05 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677792+0000", 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8170999999999999e-05 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677803+0000", 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0604999999999999e-05 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677867+0000", 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4280000000000001e-05 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.677899+0000", 2024-05-08T08:25:15.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1761000000000003e-05 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.678834+0000", 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00093493099999999996 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.678903+0000", 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9095000000000002e-05 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:6441 3.0 e27/18 3:0362a1e0:::rbd_data.119152d8614e.000000000000000f:head v 27'56, mlcod=26'54)", 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.764338+0000", 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.55760609699999997, 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001439105, 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764338+0000", 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764338+0000", 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764340+0000", 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.148e-06 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764374+0000", 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3760000000000002e-05 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764399+0000", 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4967999999999999e-05 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764411+0000", 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2541000000000001e-05 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764491+0000", 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9487000000000007e-05 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764518+0000", 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6933000000000001e-05 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.765724+0000", 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012066799999999999 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.765777+0000", 2024-05-08T08:25:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2587999999999999e-05 2024-05-08T08:25:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.404 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.404 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:15.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:15.406 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:15.413+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:15.506 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:15.514+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:15.556 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:15.556 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:15.556 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:15.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:15.608 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:15.616+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:15.675 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:15.675 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:15.675 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:15.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:15.725 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:15.732+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:15.809 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:15.817+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:15.835 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:15.835 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:15.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:15.835 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:15.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1684 3.7 3:e31dbc09:::rbd_data.119116cc7dd0.0000000000000188:head [set-alloc-hint object_size 262144 write_size 262144,write 149504~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.121640+0000", 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7199039230000004, 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054458309999999999, 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1684 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121640+0000", 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121640+0000", 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121642+0000", 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4920000000000001e-06 2024-05-08T08:25:15.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121657+0000", 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5727999999999999e-05 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121674+0000", 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6531000000000001e-05 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121683+0000", 2024-05-08T08:25:15.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1889999999999996e-06 2024-05-08T08:25:15.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121757+0000", 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3639000000000004e-05 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.121849+0000", 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2428000000000003e-05 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.122005+0000", 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000155921 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.122753+0000", 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000748374 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.126964+0000", 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042103540000000004 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127031+0000", 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7693000000000004e-05 2024-05-08T08:25:15.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.127086+0000", 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4481999999999998e-05 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1726 3.b 3:d54f3c11:::rbd_data.119116cc7dd0.00000000000002dc:head [set-alloc-hint object_size 262144 write_size 262144,write 210944~4096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.168544+0000", 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.6730003350000002, 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056432180000000002, 2024-05-08T08:25:15.844 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1726 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168544+0000", 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168544+0000", 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168545+0000", 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.668e-06 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168559+0000", 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3949e-05 2024-05-08T08:25:15.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168576+0000", 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6396e-05 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168583+0000", 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6129999999999998e-06 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168639+0000", 2024-05-08T08:25:15.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5265e-05 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168758+0000", 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000119265 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.168870+0000", 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011209699999999999 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.170243+0000", 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001373256 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174099+0000", 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038553760000000002 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174142+0000", 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3720000000000002e-05 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.174187+0000", 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4613000000000001e-05 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1925 3.7 3:e623465e:::rbd_data.119116cc7dd0.00000000000003a1:head [set-alloc-hint object_size 262144 write_size 262144,write 59392~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.346501+0000", 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4950431569999996, 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0054341750000000003, 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1925 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346501+0000", 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346501+0000", 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346502+0000", 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3099999999999996e-07 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346509+0000", 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.374e-06 2024-05-08T08:25:15.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346519+0000", 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0274e-05 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346525+0000", 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5509999999999999e-06 2024-05-08T08:25:15.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346579+0000", 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3998e-05 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346629+0000", 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0670000000000001e-05 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.346716+0000", 2024-05-08T08:25:15.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6349999999999998e-05 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.347249+0000", 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00053301200000000003 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.351865+0000", 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046158220000000003 2024-05-08T08:25:15.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.351909+0000", 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4480000000000001e-05 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.351935+0000", 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5812999999999998e-05 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1950 3.7 3:e623465e:::rbd_data.119116cc7dd0.00000000000003a1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.373087+0000", 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4684564709999997, 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050660879999999998, 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1950 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373087+0000", 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373087+0000", 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373088+0000", 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1999999999999999e-07 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373092+0000", 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4039999999999999e-06 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373101+0000", 2024-05-08T08:25:15.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9080000000000007e-06 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373107+0000", 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.981e-06 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373166+0000", 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8687000000000002e-05 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373255+0000", 2024-05-08T08:25:15.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8771999999999996e-05 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.373382+0000", 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012696300000000001 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.374231+0000", 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00084901499999999995 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.378081+0000", 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038498199999999999 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.378110+0000", 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9611000000000002e-05 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.378154+0000", 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3306999999999997e-05 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1983 3.7 3:e216bce7:::rbd_data.119116cc7dd0.00000000000000ae:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4316654099999999, 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052937829999999998, 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.863 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1983 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409879+0000", 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8100000000000001e-07 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409890+0000", 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0968000000000001e-05 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409903+0000", 2024-05-08T08:25:15.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2294e-05 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409908+0000", 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4700000000000001e-06 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409950+0000", 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2185999999999999e-05 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.409993+0000", 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2233e-05 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410050+0000", 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7778999999999997e-05 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.410519+0000", 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046863200000000002 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415079+0000", 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045602949999999998 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415132+0000", 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2516000000000003e-05 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.415172+0000", 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0528999999999998e-05 2024-05-08T08:25:15.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:1999 3.1e 3:7e7612a5:::rbd_data.119116cc7dd0.0000000000000179:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.416389+0000", 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.425154933, 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046829560000000003, 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 1999 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416389+0000", 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416389+0000", 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416390+0000", 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2699999999999999e-06 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416395+0000", 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416408+0000", 2024-05-08T08:25:15.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2519e-05 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416415+0000", 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9219999999999997e-06 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416472+0000", 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6471999999999997e-05 2024-05-08T08:25:15.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416540+0000", 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8308999999999999e-05 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.416636+0000", 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5778000000000004e-05 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.417097+0000", 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046121999999999998 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.420976+0000", 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0038786670000000001 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.421025+0000", 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8607999999999999e-05 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.421072+0000", 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7451999999999999e-05 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2111 3.18 3:1d87061b:::rbd_data.119116cc7dd0.00000000000000a4:head [set-alloc-hint object_size 262144 write_size 262144,write 200704~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.506882+0000", 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.3346621370000005, 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055828070000000004, 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.875 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2111 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506882+0000", 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506882+0000", 2024-05-08T08:25:15.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506883+0000", 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.457e-06 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506903+0000", 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9465000000000001e-05 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506919+0000", 2024-05-08T08:25:15.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6268000000000001e-05 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506928+0000", 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.738e-06 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.506980+0000", 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2748999999999999e-05 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.507079+0000", 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8148999999999998e-05 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.507167+0000", 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8405999999999993e-05 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.509392+0000", 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022244750000000001 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512382+0000", 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029903820000000002 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512418+0000", 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6390000000000002e-05 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512465+0000", 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6328000000000001e-05 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2148 3.b 3:d6d99b55:::rbd_data.119116cc7dd0.0000000000000329:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2945060169999998, 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055972469999999996, 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2148 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547038+0000", 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.13e-07 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547042+0000", 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.585e-06 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547050+0000", 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.089e-06 2024-05-08T08:25:15.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547058+0000", 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9060000000000002e-06 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547541+0000", 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048342399999999998 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547616+0000", 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4493000000000007e-05 2024-05-08T08:25:15.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547717+0000", 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100739 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552422+0000", 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047050369999999996 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552546+0000", 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012416599999999999 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552594+0000", 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8498000000000002e-05 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552635+0000", 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0797000000000003e-05 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2147 3.15 e23/18 3:aace63c7:::rbd_data.119116cc7dd0.0000000000000328:head v 23'16, mlcod=23'14)", 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2941943340000002, 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051080810000000004, 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547350+0000", 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1600000000000001e-07 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547357+0000", 2024-05-08T08:25:15.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7660000000000001e-06 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547367+0000", 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0142e-05 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547374+0000", 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8970000000000002e-06 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547427+0000", 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2701999999999998e-05 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547446+0000", 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.946e-05 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552415+0000", 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.004969238 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552458+0000", 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2259999999999999e-05 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2153 3.8 3:11eb6c23:::rbd_data.119116cc7dd0.000000000000039a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.547373+0000", 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.2941710430000004, 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0052376890000000002, 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.891 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2153 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547373+0000", 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547373+0000", 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547374+0000", 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7499999999999999e-07 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547378+0000", 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4329999999999997e-06 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547390+0000", 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1804000000000001e-05 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547398+0000", 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7819999999999994e-06 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547456+0000", 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8224999999999999e-05 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547530+0000", 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3781000000000004e-05 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.547640+0000", 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000110091 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552462+0000", 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048225810000000003 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552549+0000", 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6224999999999995e-05 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552585+0000", 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5880000000000002e-05 2024-05-08T08:25:15.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.552611+0000", 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6012000000000001e-05 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2334 3.1e 3:7c0c4f5a:::rbd_data.119116cc7dd0.0000000000000017:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1340482999999999, 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048629379999999998, 2024-05-08T08:25:15.897 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2334 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707496+0000", 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7899999999999998e-07 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707500+0000", 2024-05-08T08:25:15.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8399999999999997e-06 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707509+0000", 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5709999999999998e-06 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707515+0000", 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707572+0000", 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6699e-05 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707653+0000", 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0995999999999995e-05 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 6", 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.707826+0000", 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017260100000000001 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709844+0000", 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020187820000000002 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.712250+0000", 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002405917 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.712310+0000", 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9596999999999999e-05 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.712359+0000", 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8634999999999998e-05 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2371 3.8 3:158ebef6:::rbd_data.119116cc7dd0.0000000000000204:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1208604480000002, 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094009569999999997, 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2371 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720684+0000", 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5099999999999998e-07 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720689+0000", 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8099999999999997e-06 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720701+0000", 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.153e-05 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720710+0000", 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3640000000000005e-06 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720778+0000", 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7866000000000004e-05 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.720864+0000", 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6384999999999994e-05 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721113+0000", 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024862300000000001 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725338+0000", 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042250090000000001 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729983+0000", 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046450659999999998 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.730040+0000", 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6422999999999998e-05 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.730084+0000", 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4929999999999998e-05 2024-05-08T08:25:15.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2372 3.7 3:e6a28cad:::rbd_data.119116cc7dd0.0000000000000205:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721541+0000", 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1200030590000001, 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0056699929999999999, 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.909 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2372 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721541+0000", 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721541+0000", 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721542+0000", 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.37e-07 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721546+0000", 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2690000000000001e-06 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721558+0000", 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1883e-05 2024-05-08T08:25:15.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721565+0000", 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2749999999999998e-06 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721640+0000", 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4742999999999999e-05 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721718+0000", 2024-05-08T08:25:15.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8545000000000001e-05 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721894+0000", 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000175954 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.723729+0000", 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001834303 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727129+0000", 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034002059999999998 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727166+0000", 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7283999999999997e-05 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727211+0000", 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4793999999999998e-05 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2375 3.b 3:d52caae0:::rbd_data.119116cc7dd0.0000000000000208:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721944+0000", 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.11960031, 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053243179999999998, 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2375 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.915 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721944+0000", 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721944+0000", 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721945+0000", 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.64e-07 2024-05-08T08:25:15.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721949+0000", 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4989999999999996e-06 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721961+0000", 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2232000000000001e-05 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721972+0000", 2024-05-08T08:25:15.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0436e-05 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722039+0000", 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7657999999999995e-05 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722115+0000", 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5370000000000005e-05 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:15.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722275+0000", 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015985000000000001 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725627+0000", 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033519410000000002 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727159+0000", 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001531969 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727212+0000", 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3507000000000001e-05 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.727268+0000", 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5892e-05 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2377 3.8 3:17a99b47:::rbd_data.119116cc7dd0.000000000000038a:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:15.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.722517+0000", 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1190272349999999, 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01043969, 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2377 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722517+0000", 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722517+0000", 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722518+0000", 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4099999999999997e-07 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722523+0000", 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8960000000000002e-06 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722534+0000", 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0994e-05 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722541+0000", 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3599999999999998e-06 2024-05-08T08:25:15.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722580+0000", 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9405e-05 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722642+0000", 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1231999999999999e-05 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:15.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.722762+0000", 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000120426 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.730110+0000", 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073483749999999999 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732831+0000", 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002720685 2024-05-08T08:25:15.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732897+0000", 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6106999999999996e-05 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.732956+0000", 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9268999999999998e-05 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:2713 3.2 3:42c14b71:::rbd_data.1191a2472683.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 22192118~19628] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:12.030868+0000", 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 3.8106756640000001, 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098761920000000007, 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 2713 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.927 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.030868+0000", 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.030868+0000", 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.030870+0000", 2024-05-08T08:25:15.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.232e-06 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.032336+0000", 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001466443 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.032356+0000", 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0183e-05 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.032368+0000", 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1861999999999999e-05 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.032444+0000", 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5883000000000005e-05 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.032629+0000", 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018550199999999999 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.033087+0000", 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000457582 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.039318+0000", 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006231215 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.040660+0000", 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013420179999999999 2024-05-08T08:25:15.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.040703+0000", 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2632000000000001e-05 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:12.040744+0000", 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1640000000000001e-05 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.932 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6425 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.132721138, 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03287702, 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6425 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708825+0000", 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7e-06 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708833+0000", 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708851+0000", 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7961e-05 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708859+0000", 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2439999999999993e-06 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708920+0000", 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1178000000000003e-05 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.709031+0000", 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011102 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.714353+0000", 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053222440000000003 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741561+0000", 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027207346 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741581+0000", 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0854000000000001e-05 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741644+0000", 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2526000000000004e-05 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741700+0000", 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5812e-05 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4497.0:6425 3.7 e27/18 ondisk, result = 0)", 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.733408+0000", 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.108135917, 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0081620400000000006, 2024-05-08T08:25:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.733408+0000", 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.733408+0000", 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.733409+0000", 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3740000000000001e-06 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.733417+0000", 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.294e-06 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.733434+0000", 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7776000000000001e-05 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.733444+0000", 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7939999999999997e-06 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741542+0000", 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080977540000000004 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741553+0000", 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1097999999999999e-05 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741570+0000", 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6949999999999999e-05 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6447 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.067287388, 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020412153999999998, 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6447 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774258+0000", 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774267+0000", 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8869999999999998e-06 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774285+0000", 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7399000000000001e-05 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774293+0000", 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1089999999999998e-06 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774350+0000", 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774416+0000", 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6166000000000003e-05 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.777145+0000", 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027283160000000002 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794536+0000", 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017391163000000001 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794555+0000", 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9377000000000001e-05 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794614+0000", 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8829000000000002e-05 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794669+0000", 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop_reply(client.4497.0:6447 3.7 e27/18 ondisk, result = 0)", 2024-05-08T08:25:15.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.787067+0000", 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0544771559999999, 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074796330000000003, 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.787067+0000", 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.787067+0000", 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.787068+0000", 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2240000000000001e-06 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.787074+0000", 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.66e-06 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.787089+0000", 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5180999999999999e-05 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.787098+0000", 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9160000000000007e-06 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794516+0000", 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0074182939999999998 2024-05-08T08:25:15.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794528+0000", 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1946999999999999e-05 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794546+0000", 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8411e-05 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:15.953 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:15.955 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:15.956 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:15.963+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:15.968 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:25:16.053 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone4 from 0xb97d171 (overlap 0x1bcfaef) to 0x58fefc 2024-05-08T08:25:16.056 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:16.063+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:16.078 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:16.078 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:16.078 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:16.078 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:16.087 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:16.138 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-05-08T08:25:16.139 INFO:teuthology.orchestra.run.smithi031.stdout:671 write 0x80ab553 thru 0x80ac636 (0x10e4 bytes) 2024-05-08T08:25:16.157 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:16.164+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:16.157 INFO:teuthology.orchestra.run.smithi031.stdout:672 read 0x60d276b thru 0x60d3fa4 (0x183a bytes) 2024-05-08T08:25:16.158 INFO:teuthology.orchestra.run.smithi031.stdout:673 trunc from 0xb97d171 to 0xca6374c 2024-05-08T08:25:16.177 INFO:teuthology.orchestra.run.smithi031.stdout:674 trunc from 0xca6374c to 0xd7e48f 2024-05-08T08:25:16.202 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:16.202 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:16.202 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:16.202 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:16.202 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:16.215 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:16.221 INFO:teuthology.orchestra.run.smithi031.stdout:677 punch from 0x327a7c to 0x33293b, (0xaebf bytes) 2024-05-08T08:25:16.222 INFO:teuthology.orchestra.run.smithi031.stdout:678 read 0x2b688b thru 0x2bc61d (0x5d93 bytes) 2024-05-08T08:25:16.229 INFO:teuthology.orchestra.run.smithi031.stdout:680 writesame 0x207bda8 thru 0x20866a8 data_size 0x11(0xa901 bytes) 2024-05-08T08:25:16.230 INFO:teuthology.orchestra.run.smithi031.stdout:681 read 0x138ff9b thru 0x139015f (0x1c5 bytes) 2024-05-08T08:25:16.236 INFO:teuthology.orchestra.run.smithi031.stdout:682 trunc from 0x20866a9 to 0x6cba894 2024-05-08T08:25:16.244 INFO:teuthology.orchestra.run.smithi031.stdout:683 write 0x4cb12e4 thru 0x4cc0a1c (0xf739 bytes) 2024-05-08T08:25:16.245 INFO:teuthology.orchestra.run.smithi031.stdout:684 write 0x4e97926 thru 0x4ea60c8 (0xe7a3 bytes) 2024-05-08T08:25:16.247 INFO:teuthology.orchestra.run.smithi031.stdout:685 write 0x1d129cb thru 0x1d16e94 (0x44ca bytes) 2024-05-08T08:25:16.248 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:16.248 INFO:teuthology.orchestra.run.smithi031.stdout:687 read 0x6200b45 thru 0x62039b8 (0x2e74 bytes) 2024-05-08T08:25:16.249 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:16.252 INFO:teuthology.orchestra.run.smithi031.stdout:689 trunc from 0x6cba894 to 0x4092c89 2024-05-08T08:25:16.258 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:16.265+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:16.263 INFO:teuthology.orchestra.run.smithi031.stdout:690 writesame 0x32f1214 thru 0x32fea65 data_size 0x1(0xd852 bytes) 2024-05-08T08:25:16.297 INFO:teuthology.orchestra.run.smithi031.stdout:692 writesame 0x745aa22 thru 0x745b88e data_size 0x1(0xe6d bytes) 2024-05-08T08:25:16.298 INFO:teuthology.orchestra.run.smithi031.stdout:693 punch from 0x67f1dda to 0x68011de, (0xf404 bytes) 2024-05-08T08:25:16.312 INFO:teuthology.orchestra.run.smithi031.stdout:694 writesame 0xe0a4eb6 thru 0xe0b19a6 data_size 0x1(0xcaf1 bytes) 2024-05-08T08:25:16.314 INFO:teuthology.orchestra.run.smithi031.stdout:695 write 0x110ddd1 thru 0x1115e18 (0x8048 bytes) 2024-05-08T08:25:16.315 INFO:teuthology.orchestra.run.smithi031.stdout:696 punch from 0x1c8415 to 0x1d6d00, (0xe8eb bytes) 2024-05-08T08:25:16.315 INFO:teuthology.orchestra.run.smithi031.stdout:697 read 0x4c2ef8c thru 0x4c38e74 (0x9ee9 bytes) 2024-05-08T08:25:16.316 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:16.316 INFO:teuthology.orchestra.run.smithi031.stdout:700 read 0x2f864ff thru 0x2f8dd4c (0x784e bytes) 2024-05-08T08:25:16.316 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:16.316 INFO:teuthology.orchestra.run.smithi031.stdout:704 punch from 0x865af28 to 0x866876c, (0xd844 bytes) 2024-05-08T08:25:16.321 INFO:teuthology.orchestra.run.smithi031.stdout:705 writesame 0xe3ef3f7 thru 0xe3f5954 data_size 0x4b(0x655e bytes) 2024-05-08T08:25:16.323 INFO:teuthology.orchestra.run.smithi031.stdout:706 writesame 0x8e5a4b0 thru 0x8e63a88 data_size 0x13(0x95d9 bytes) 2024-05-08T08:25:16.325 INFO:teuthology.orchestra.run.smithi031.stdout:708 read 0x9c8d2d9 thru 0x9c9185f (0x4587 bytes) 2024-05-08T08:25:16.325 INFO:teuthology.orchestra.run.smithi031.stdout:710 write 0x7bfa17b thru 0x7c02891 (0x8717 bytes) 2024-05-08T08:25:16.326 INFO:teuthology.orchestra.run.smithi031.stdout:711 read 0x8a7b3c0 thru 0x8a86284 (0xaec5 bytes) 2024-05-08T08:25:16.327 INFO:teuthology.orchestra.run.smithi031.stdout:713 write 0xbfcae04 thru 0xbfd1cfb (0x6ef8 bytes) 2024-05-08T08:25:16.328 INFO:teuthology.orchestra.run.smithi031.stdout:714 punch from 0x70db3ec to 0x70dce3b, (0x1a4f bytes) 2024-05-08T08:25:16.329 INFO:teuthology.orchestra.run.smithi031.stdout:715 write 0xd5523a5 thru 0xd55b624 (0x9280 bytes) 2024-05-08T08:25:16.330 INFO:teuthology.orchestra.run.smithi031.stdout:716 write 0x4fec27e thru 0x4ffaf04 (0xec87 bytes) 2024-05-08T08:25:16.332 INFO:teuthology.orchestra.run.smithi031.stdout:717 clone 6 order 25 su 4194304 sc 13 2024-05-08T08:25:16.362 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:16.366+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:16.367 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.7621758599999993, 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098847239999999992, 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 9 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:16.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611387+0000", 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.181e-06 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611395+0000", 2024-05-08T08:25:16.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0730000000000005e-06 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611411+0000", 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6634999999999999e-05 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611427+0000", 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5699e-05 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611486+0000", 2024-05-08T08:25:16.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8692e-05 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611491+0000", 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.989e-06 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615250+0000", 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037591579999999999 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615342+0000", 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2561999999999999e-05 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615480+0000", 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013755399999999999 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.616631+0000", 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011506999999999999 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621178+0000", 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045473620000000001 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621230+0000", 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1845999999999998e-05 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621269+0000", 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9273000000000002e-05 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.373 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1683 3.e e23/18 3:777f8546:::rbd_data.119116cc7dd0.0000000000000187:head v 23'35, mlcod=23'33)", 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.2528442650000002, 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059551170000000002, 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:16.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120725+0000", 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3829999999999992e-06 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120734+0000", 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7099999999999996e-06 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120739+0000", 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120789+0000", 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0203e-05 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120806+0000", 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7085999999999999e-05 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126597+0000", 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005790936 2024-05-08T08:25:16.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126671+0000", 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3837999999999993e-05 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1680 3.1b e23/18 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head v 23'453, mlcod=23'449)", 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.2527821860000001, 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059238520000000003, 2024-05-08T08:25:16.378 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120779+0000", 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120784+0000", 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5809999999999996e-06 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120802+0000", 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8063999999999998e-05 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120809+0000", 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6630000000000004e-06 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120863+0000", 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3918000000000001e-05 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120878+0000", 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4822e-05 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126657+0000", 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057795650000000004 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126702+0000", 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4610000000000001e-05 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1826 3.3 3:c3deaffe:::rbd_data.119116cc7dd0.00000000000001e1:head [set-alloc-hint object_size 262144 write_size 262144,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.382 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.1221369880000003, 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0079090730000000008, 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1826 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:16.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251425+0000", 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.215e-06 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251433+0000", 2024-05-08T08:25:16.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9060000000000008e-06 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251444+0000", 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0813e-05 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251449+0000", 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251489+0000", 2024-05-08T08:25:16.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9774999999999998e-05 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251542+0000", 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3162999999999999e-05 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251630+0000", 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7309e-05 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.252128+0000", 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00049835099999999996 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259274+0000", 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071460289999999999 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259308+0000", 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3679000000000001e-05 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259332+0000", 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4752999999999998e-05 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1899 3.1b e23/18 3:dbe9d447:::rbd_data.119116cc7dd0.0000000000000372:head v 23'532, mlcod=23'528)", 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.325037+0000", 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.0485231239999999, 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0055009619999999999, 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.388 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325037+0000", 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325037+0000", 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325038+0000", 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325054+0000", 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5925000000000001e-05 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325062+0000", 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3780000000000001e-06 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325067+0000", 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7040000000000002e-06 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325094+0000", 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6855999999999998e-05 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.325104+0000", 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0882e-05 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330511+0000", 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054067309999999997 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330538+0000", 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7042e-05 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1919 3.1b e23/18 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head v 23'539, mlcod=23'533)", 2024-05-08T08:25:16.392 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.340621+0000", 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.0329397629999999, 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054933569999999999, 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340621+0000", 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340621+0000", 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340621+0000", 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340630+0000", 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7749999999999998e-06 2024-05-08T08:25:16.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340639+0000", 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2220000000000008e-06 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340644+0000", 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4800000000000003e-06 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340687+0000", 2024-05-08T08:25:16.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3086e-05 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.340706+0000", 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9402000000000002e-05 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.346075+0000", 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053688640000000001 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.346114+0000", 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9014000000000003e-05 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1976 3.3 3:c4f5f4db:::rbd_data.119116cc7dd0.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.9653155340000001, 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065238700000000002, 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.397 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1976 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:16.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.39e-07 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408247+0000", 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8729999999999999e-06 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408251+0000", 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.357e-06 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408254+0000", 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9699999999999999e-06 2024-05-08T08:25:16.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408768+0000", 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00051321699999999995 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408852+0000", 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4888000000000004e-05 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:16.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408963+0000", 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110176 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409769+0000", 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00080602600000000005 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414684+0000", 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049152529999999996 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414731+0000", 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.702e-05 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414769+0000", 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7750999999999998e-05 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1972 3.1d e23/18 3:ba47efeb:::rbd_data.119116cc7dd0.0000000000000037:head v 23'32, mlcod=23'30)", 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.9649839599999996, 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0061126660000000001, 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:16.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408577+0000", 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3600000000000003e-07 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408583+0000", 2024-05-08T08:25:16.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7139999999999998e-06 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408593+0000", 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0429e-05 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408600+0000", 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408870+0000", 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00027024599999999997 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408889+0000", 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8961999999999999e-05 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414654+0000", 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057644200000000001 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414689+0000", 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5537999999999998e-05 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1979 3.1b e23/18 3:d8488a8a:::rbd_data.119116cc7dd0.0000000000000086:head v 23'552, mlcod=23'550)", 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.9645798169999997, 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058253389999999997, 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.408 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7599999999999997e-07 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408987+0000", 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5149999999999997e-06 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408996+0000", 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.38e-06 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409002+0000", 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1979999999999998e-06 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409049+0000", 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6844000000000001e-05 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409067+0000", 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7629999999999999e-05 2024-05-08T08:25:16.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414772+0000", 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057050579999999998 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414806+0000", 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4137999999999997e-05 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.412 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2003 3.14 3:2837a914:::rbd_data.119116cc7dd0.00000000000001e3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.417654+0000", 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.9559067470000002, 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0055974170000000004, 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2003 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417654+0000", 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417654+0000", 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417654+0000", 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5700000000000002e-07 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417661+0000", 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0169999999999999e-06 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417671+0000", 2024-05-08T08:25:16.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3139999999999993e-06 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417676+0000", 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8670000000000002e-06 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417719+0000", 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3739999999999998e-05 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417767+0000", 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7827000000000001e-05 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417825+0000", 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7992999999999999e-05 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.422918+0000", 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050932369999999996 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423181+0000", 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00026219399999999999 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423215+0000", 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4050000000000001e-05 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423251+0000", 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6421000000000002e-05 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2145 3.1d e23/18 3:bdb88d54:::rbd_data.119116cc7dd0.0000000000000326:head v 23'39, mlcod=23'36)", 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.547581+0000", 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8259793359999996, 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0055608239999999998, 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547581+0000", 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547581+0000", 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547582+0000", 2024-05-08T08:25:16.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2500000000000005e-07 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547587+0000", 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2950000000000001e-06 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547614+0000", 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6954999999999999e-05 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547628+0000", 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4046000000000001e-05 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547699+0000", 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1487000000000002e-05 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547720+0000", 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0373999999999998e-05 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553058+0000", 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053383830000000004 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553142+0000", 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3659000000000007e-05 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2224 3.1b e23/18 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head v 23'619, mlcod=23'614)", 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.620074+0000", 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.7534861360000003, 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0055822069999999996, 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620074+0000", 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620074+0000", 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620075+0000", 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620080+0000", 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2730000000000002e-06 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620091+0000", 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0691e-05 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620097+0000", 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5699999999999998e-06 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620130+0000", 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2227999999999997e-05 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.620141+0000", 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1846e-05 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.625612+0000", 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054707230000000003 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.625656+0000", 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4202000000000001e-05 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2374 3.1a 3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.6522448179999998, 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011630434, 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.428 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2374 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1349999999999999e-06 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721322+0000", 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.13e-06 2024-05-08T08:25:16.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721340+0000", 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7988000000000001e-05 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721348+0000", 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9789999999999993e-06 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721408+0000", 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0168999999999999e-05 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721494+0000", 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6001000000000006e-05 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721716+0000", 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022242200000000001 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.722577+0000", 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00086067299999999995 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732885+0000", 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010308286 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732916+0000", 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0437999999999999e-05 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732946+0000", 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0213e-05 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2386 3.1b e24/18 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head v 24'634, mlcod=24'630)", 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.737973+0000", 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.6355874569999997, 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056054, 2024-05-08T08:25:16.434 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.737973+0000", 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.737973+0000", 2024-05-08T08:25:16.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.737973+0000", 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.737978+0000", 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7280000000000003e-06 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.737987+0000", 2024-05-08T08:25:16.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4020000000000002e-06 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.737992+0000", 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3870000000000003e-06 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.738057+0000", 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5094999999999999e-05 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.738071+0000", 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4177e-05 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.743537+0000", 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054658730000000004 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.743578+0000", 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1174999999999998e-05 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:6914 3.13 e28/18 3:cdf44573:::rbd_data.119120fdbc10.0000000000000004:head v 28'114, mlcod=26'112)", 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.159595+0000", 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.213965197, 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054549940000000003, 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.439 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159595+0000", 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159595+0000", 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159596+0000", 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1769999999999999e-06 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.161525+0000", 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0019288090000000001 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.161547+0000", 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2268000000000001e-05 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.161560+0000", 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2427e-05 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.161648+0000", 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8437999999999996e-05 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.161675+0000", 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6971e-05 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.164992+0000", 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033167549999999998 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165050+0000", 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8149000000000002e-05 2024-05-08T08:25:16.443 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7094 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1565839~19667825] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.168348584, 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008628854, 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7094 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205213+0000", 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.54e-07 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206767+0000", 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001554607 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206779+0000", 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1599e-05 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206785+0000", 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0050000000000001e-06 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206831+0000", 2024-05-08T08:25:16.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6108e-05 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206916+0000", 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5575000000000007e-05 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207099+0000", 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000182237 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209859+0000", 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027605020000000002 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213776+0000", 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003917238 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213810+0000", 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4051999999999999e-05 2024-05-08T08:25:16.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213841+0000", 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0176999999999999e-05 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7095 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:16.450 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.16676174499999999, 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071300790000000001, 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7095 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206801+0000", 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3429999999999998e-06 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206810+0000", 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8389999999999996e-06 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206814+0000", 2024-05-08T08:25:16.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1119999999999998e-06 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208857+0000", 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020428310000000002 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208905+0000", 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7849999999999998e-05 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209120+0000", 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000215362 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209862+0000", 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074158700000000004 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213871+0000", 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040092579999999999 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213903+0000", 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1661000000000001e-05 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213929+0000", 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5795e-05 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.456 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7096 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.16625473099999999, 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072441390000000001, 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.457 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7096 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207309+0000", 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6960000000000001e-06 2024-05-08T08:25:16.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207315+0000", 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207319+0000", 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8840000000000004e-06 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209378+0000", 2024-05-08T08:25:16.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020587029999999998 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209422+0000", 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4320000000000003e-05 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209632+0000", 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020937899999999999 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.210465+0000", 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083359500000000002 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214459+0000", 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039939270000000004 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214514+0000", 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4338000000000001e-05 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214550+0000", 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6130000000000001e-05 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:16.463 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.102183239, 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:25:16.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:16.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:7160 3.1a e28/18 ondisk, result = 0)", 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:16.469 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.086650956000000001, 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0078836489999999995, 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:16.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286911+0000", 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4580000000000001e-06 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286918+0000", 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9750000000000001e-06 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286935+0000", 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7337e-05 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286946+0000", 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0611999999999999e-05 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294654+0000", 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.007707958 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294664+0000", 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0475999999999999e-05 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294793+0000", 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012883299999999999 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.473 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:16.475 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:16.475 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:16.481+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:16.575 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:16.583+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:16.622 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:16.622 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:16.622 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:16.622 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:16.630 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:16.676 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:16.683+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:16.735 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:16.735 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:16.735 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:16.736 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:16.736 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:16.744 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:16.763 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:16.806 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:16.814+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.852 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1700 3.1d 3:bbece77f:::rbd_data.119116cc7dd0.0000000000000263:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7188901139999997, 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049688420000000002, 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1700 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3900000000000004e-07 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139611+0000", 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7419999999999999e-06 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139621+0000", 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6369999999999994e-06 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139636+0000", 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5367e-05 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139684+0000", 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899000000000003e-05 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139733+0000", 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9205000000000001e-05 2024-05-08T08:25:16.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139798+0000", 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4431999999999996e-05 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144321+0000", 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045238159999999999 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144511+0000", 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018977100000000001 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144544+0000", 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2620000000000003e-05 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144575+0000", 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1714000000000002e-05 2024-05-08T08:25:16.858 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1699 3.1f e23/18 3:fdba1daf:::rbd_data.119116cc7dd0.0000000000000262:head v 23'23, mlcod=23'21)", 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.139891+0000", 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7186061239999999, 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0046779760000000004, 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139891+0000", 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139891+0000", 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139891+0000", 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8299999999999996e-07 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139896+0000", 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0760000000000002e-06 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139904+0000", 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9209999999999992e-06 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139909+0000", 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5510000000000001e-06 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139966+0000", 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7046999999999998e-05 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139983+0000", 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7105999999999999e-05 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144541+0000", 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045576159999999996 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144569+0000", 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7976000000000001e-05 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1814 3.1c 3:39f766e8:::rbd_data.119116cc7dd0.0000000000000145:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.239933+0000", 2024-05-08T08:25:16.863 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.6185637010000002, 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044561519999999997, 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1814 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.239933+0000", 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.239933+0000", 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.239934+0000", 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.023e-06 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.239939+0000", 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.938e-06 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.239950+0000", 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1198e-05 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.239957+0000", 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5880000000000003e-06 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.240008+0000", 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1119999999999998e-05 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.240095+0000", 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7208999999999998e-05 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.240802+0000", 2024-05-08T08:25:16.867 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00070732900000000003 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244255+0000", 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0034525810000000001 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244296+0000", 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1474999999999998e-05 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244348+0000", 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1530000000000003e-05 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244389+0000", 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1161000000000001e-05 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1933 3.1d 3:ba47efeb:::rbd_data.119116cc7dd0.0000000000000037:head [set-alloc-hint object_size 262144 write_size 262144,write 219136~10240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.355573+0000", 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.5029241600000001, 2024-05-08T08:25:16.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045565090000000003, 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1933 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355573+0000", 2024-05-08T08:25:16.870 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355573+0000", 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355574+0000", 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5099999999999998e-07 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355589+0000", 2024-05-08T08:25:16.871 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5722999999999998e-05 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355603+0000", 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4149e-05 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355610+0000", 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.387e-06 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355673+0000", 2024-05-08T08:25:16.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3145999999999995e-05 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355806+0000", 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000133454 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.355950+0000", 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00014379400000000001 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.357392+0000", 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001442193 2024-05-08T08:25:16.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.360053+0000", 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002661133 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.360101+0000", 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7676999999999998e-05 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.360129+0000", 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7902000000000002e-05 2024-05-08T08:25:16.874 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1972 3.1d 3:ba47efeb:::rbd_data.119116cc7dd0.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.4502477950000001, 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066697570000000001, 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.875 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1972 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:16.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408250+0000", 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.026e-06 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408256+0000", 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8490000000000002e-06 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408268+0000", 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1827999999999999e-05 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408275+0000", 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408331+0000", 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5989000000000002e-05 2024-05-08T08:25:16.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408406+0000", 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5666000000000004e-05 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408509+0000", 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010266400000000001 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.410688+0000", 2024-05-08T08:25:16.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002179112 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414827+0000", 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041387699999999999 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414878+0000", 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1480999999999997e-05 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414919+0000", 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0188999999999998e-05 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1989 3.1c 3:39f766e8:::rbd_data.119116cc7dd0.0000000000000145:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.446120788, 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005150277, 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.881 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1989 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412377+0000", 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3600000000000003e-07 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412380+0000", 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7639999999999999e-06 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412390+0000", 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9729999999999992e-06 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412396+0000", 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2519999999999999e-06 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412447+0000", 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1177999999999997e-05 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412525+0000", 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8375999999999996e-05 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412612+0000", 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7106000000000003e-05 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415079+0000", 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024662569999999999 2024-05-08T08:25:16.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417466+0000", 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023874170000000002 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417498+0000", 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2215000000000002e-05 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417526+0000", 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8002999999999999e-05 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1994 3.5 3:a2dbaef4:::rbd_data.119116cc7dd0.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.4432647190000001, 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052774390000000001, 2024-05-08T08:25:16.887 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1994 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415233+0000", 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.61e-07 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415238+0000", 2024-05-08T08:25:16.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7309999999999998e-06 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415250+0000", 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.118e-05 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415256+0000", 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.122e-06 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415308+0000", 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2111000000000003e-05 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415389+0000", 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0889000000000004e-05 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415492+0000", 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000103316 2024-05-08T08:25:16.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416315+0000", 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000822804 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420358+0000", 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004043128 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420470+0000", 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000112197 2024-05-08T08:25:16.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420509+0000", 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9199999999999997e-05 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1995 3.17 e23/18 3:eb8d97a8:::rbd_data.119116cc7dd0.0000000000000175:head v 23'20, mlcod=23'18)", 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.4427608650000003, 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072714820000000001, 2024-05-08T08:25:16.893 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9200000000000001e-07 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415742+0000", 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415751+0000", 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2890000000000003e-06 2024-05-08T08:25:16.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415757+0000", 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0190000000000003e-06 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415804+0000", 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7141999999999997e-05 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415824+0000", 2024-05-08T08:25:16.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0136000000000002e-05 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.422967+0000", 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071422839999999996 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423007+0000", 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0720000000000003e-05 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2002 3.1f e23/18 3:fbb43922:::rbd_data.119116cc7dd0.00000000000001e2:head v 23'25, mlcod=23'23)", 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.4406055860000002, 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051483420000000002, 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417892+0000", 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417896+0000", 2024-05-08T08:25:16.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9339999999999999e-06 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417905+0000", 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8379999999999997e-06 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417914+0000", 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9509999999999995e-06 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417969+0000", 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4225999999999999e-05 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417988+0000", 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9318999999999999e-05 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423010+0000", 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050221659999999998 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423039+0000", 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9417999999999999e-05 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2008 3.5 3:a3d8901a:::rbd_data.119116cc7dd0.0000000000000254:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.420846+0000", 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.4376509039999998, 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049252009999999997, 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.902 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2008 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420846+0000", 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420846+0000", 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420847+0000", 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420851+0000", 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0330000000000002e-06 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420861+0000", 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9799999999999993e-06 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420866+0000", 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0950000000000003e-06 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420926+0000", 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0590000000000001e-05 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421006+0000", 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.928e-05 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421099+0000", 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3850000000000004e-05 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421938+0000", 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083869299999999995 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425680+0000", 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037421989999999999 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425731+0000", 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1073999999999999e-05 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425771+0000", 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9669000000000003e-05 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.907 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2011 3.16 3:6eef803b:::rbd_data.119116cc7dd0.0000000000000257:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.4375451980000005, 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050707879999999997, 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.908 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2011 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1299999999999998e-07 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420957+0000", 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170000000000001e-06 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420969+0000", 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1938e-05 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420976+0000", 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3200000000000002e-06 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421035+0000", 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8696999999999997e-05 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421080+0000", 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4709000000000002e-05 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421139+0000", 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9762999999999997e-05 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425663+0000", 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045239520000000004 2024-05-08T08:25:16.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425971+0000", 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00030778899999999999 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425999+0000", 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8005e-05 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.426022+0000", 2024-05-08T08:25:16.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3184999999999999e-05 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2126 3.16 3:6a294600:::rbd_data.119116cc7dd0.000000000000015b:head [set-alloc-hint object_size 262144 write_size 262144,write 59392~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.337599258, 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066558700000000004, 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.914 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2126 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520898+0000", 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7999999999999993e-07 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520908+0000", 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2520000000000005e-06 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520920+0000", 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1891e-05 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520927+0000", 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4950000000000002e-06 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520980+0000", 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2895000000000001e-05 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521065+0000", 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4698999999999997e-05 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:16.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521193+0000", 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012812199999999999 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521826+0000", 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063310000000000005 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527495+0000", 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056689669999999996 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527529+0000", 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4394000000000003e-05 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527553+0000", 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4074999999999999e-05 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2133 3.16 3:6a294600:::rbd_data.119116cc7dd0.000000000000015b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.3182860889999999, 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065993600000000003, 2024-05-08T08:25:16.919 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2133 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7800000000000001e-07 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540215+0000", 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7689999999999998e-06 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540225+0000", 2024-05-08T08:25:16.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0039e-05 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540230+0000", 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2830000000000001e-06 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540280+0000", 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9480000000000001e-05 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540352+0000", 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2347000000000004e-05 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540409+0000", 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6518999999999999e-05 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541006+0000", 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00059697199999999998 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546717+0000", 2024-05-08T08:25:16.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057116579999999997 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546764+0000", 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7039000000000001e-05 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546810+0000", 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5575999999999999e-05 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.924 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2140 3.5 3:a1b38341:::rbd_data.119116cc7dd0.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.3166447269999999, 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00497986, 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2140 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541853+0000", 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541857+0000", 2024-05-08T08:25:16.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5469999999999998e-06 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541868+0000", 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0505e-05 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541873+0000", 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0989999999999998e-06 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541923+0000", 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0383000000000002e-05 2024-05-08T08:25:16.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542000+0000", 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6960999999999997e-05 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542106+0000", 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000105717 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542543+0000", 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00043726399999999998 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042138540000000004 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546793+0000", 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6269000000000001e-05 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546832+0000", 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8464000000000003e-05 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.929 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2145 3.1d 3:bdb88d54:::rbd_data.119116cc7dd0.0000000000000326:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.3117401879999999, 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006666384, 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2145 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.930 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2e-07 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546760+0000", 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8990000000000001e-06 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546766+0000", 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5390000000000004e-06 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546773+0000", 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.19e-06 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547312+0000", 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053938900000000004 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547380+0000", 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8189999999999996e-05 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547474+0000", 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3450999999999997e-05 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.550283+0000", 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002809228 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553330+0000", 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030471249999999999 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553378+0000", 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7710000000000002e-05 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553423+0000", 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5142999999999998e-05 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2154 3.d e23/18 3:b40d5f62:::rbd_data.119116cc7dd0.000000000000039b:head v 23'28, mlcod=23'26)", 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.548514+0000", 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.3099826969999997, 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048007409999999999, 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.935 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548514+0000", 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548514+0000", 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548515+0000", 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999997e-07 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548520+0000", 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2270000000000001e-06 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548531+0000", 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0763e-05 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548537+0000", 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6270000000000002e-06 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548579+0000", 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.125e-05 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548592+0000", 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3553999999999999e-05 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553275+0000", 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0046825360000000002 2024-05-08T08:25:16.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553315+0000", 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0145999999999998e-05 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2197 3.1d 3:be511a0a:::rbd_data.119116cc7dd0.0000000000000209:head [set-alloc-hint object_size 262144 write_size 262144,write 4096~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.2683861929999996, 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004962401, 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.939 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2197 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590112+0000", 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0419999999999999e-06 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590120+0000", 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5050000000000007e-06 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590131+0000", 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.077e-05 2024-05-08T08:25:16.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590136+0000", 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590180+0000", 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3299e-05 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590259+0000", 2024-05-08T08:25:16.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8848000000000001e-05 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590343+0000", 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4328999999999998e-05 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.592143+0000", 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001800248 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595017+0000", 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002873553 2024-05-08T08:25:16.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595049+0000", 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2481999999999998e-05 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595073+0000", 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.37e-05 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2338 3.16 3:6f361ecb:::rbd_data.119116cc7dd0.000000000000001b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.1509793989999997, 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052194750000000003, 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2338 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707518+0000", 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707525+0000", 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7900000000000002e-06 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707549+0000", 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3414999999999999e-05 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707558+0000", 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1600000000000004e-06 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707639+0000", 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0772000000000005e-05 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707722+0000", 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3366000000000001e-05 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707911+0000", 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000189447 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.709463+0000", 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001551621 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712639+0000", 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031764419999999998 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712693+0000", 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3402000000000001e-05 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712737+0000", 2024-05-08T08:25:16.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3897999999999999e-05 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2713 3.2 e24/18 3:42c14b71:::rbd_data.1191a2472683.0000000000000000:head v 24'16, mlcod=23'14)", 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.8252095590000001, 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058213900000000001, 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033289+0000", 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036196+0000", 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0029067730000000001 2024-05-08T08:25:16.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036213+0000", 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7394e-05 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036228+0000", 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4841000000000001e-05 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036289+0000", 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1091000000000002e-05 2024-05-08T08:25:16.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036310+0000", 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0951e-05 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.039072+0000", 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002761763 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.039109+0000", 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6656999999999998e-05 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.953 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2782 3.1c 3:3afc57c8:::rbd_data.1191a2472683.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23855104~19711] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.8017681520000002, 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010424643000000001, 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:16.954 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2782 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:16.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056730+0000", 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9699999999999994e-07 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061776+0000", 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050462679999999996 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061789+0000", 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3358e-05 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:16.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061795+0000", 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1349999999999997e-06 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061844+0000", 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8414000000000001e-05 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061957+0000", 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000112759 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.062210+0000", 2024-05-08T08:25:16.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000253918 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.066162+0000", 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039516179999999996 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067075+0000", 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00091328600000000002 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067116+0000", 2024-05-08T08:25:16.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0599999999999998e-05 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067153+0000", 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7289999999999997e-05 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:16.959 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:16.960 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:16.960 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:16.961 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:16.961 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:16.961 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:16.962 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:16.967+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:16.962 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:16.962 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:16.963 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:16.963 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:16.984 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:16.985 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:17.003 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:17.003 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:17.021 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:17.021 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:17.056 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone5 intact 2024-05-08T08:25:17.061 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:17.068+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:17.110 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:17.110 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:17.119 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:17.161 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:17.169+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:17.228 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:17.228 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:17.237 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:17.262 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-05-08T08:25:17.262 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:17.270+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:17.275 INFO:teuthology.orchestra.run.smithi031.stdout:718 read 0x7a47a9c thru 0x7a56c92 (0xf1f7 bytes) 2024-05-08T08:25:17.277 INFO:teuthology.orchestra.run.smithi031.stdout:719 clone 7 order 18 su 32768 sc 6 2024-05-08T08:25:17.344 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:17.345 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:17.353 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:17.363 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:17.370+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:17.460 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:17.460 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:17.463 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:17.470+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:17.469 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:17.564 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:17.571+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:17.616 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:17.616 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:17.625 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:17.665 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:17.673+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:17.766 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:17.774+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:17.771 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:17.771 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:17.780 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:17.867 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:17.874+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:17.895 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:17.895 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:17.904 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:17.968 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:17.975+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:17.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:25:18.020 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:18.020 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:18.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:18.071 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:18.078+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:18.071 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone6 from 0xe3f5955 (overlap 0xe3f5955) to 0x5b2f088 2024-05-08T08:25:18.169 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:18.177+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:18.205 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:18.206 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:18.216 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:18.270 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:18.277+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:18.276 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-05-08T08:25:18.371 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:18.378+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:18.379 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:18.379 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:18.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:18.472 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:18.479+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:18.528 INFO:teuthology.orchestra.run.smithi031.stdout:724 punch from 0xb9a5cfb to 0xb9aaafa, (0x4dff bytes) 2024-05-08T08:25:18.532 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.532 INFO:teuthology.orchestra.run.smithi031.stdout:726 punch from 0x2fa718d to 0x2fa9cc4, (0x2b37 bytes) 2024-05-08T08:25:18.532 INFO:teuthology.orchestra.run.smithi031.stdout:727 write 0xd603ea3 thru 0xd6052df (0x143d bytes) 2024-05-08T08:25:18.540 INFO:teuthology.orchestra.run.smithi031.stdout:729 trunc from 0xe3f5955 to 0xa9b4435 2024-05-08T08:25:18.546 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:18.546 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:18.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:18.560 INFO:teuthology.orchestra.run.smithi031.stdout:732 read 0x4a0a990 thru 0x4a1975e (0xedcf bytes) 2024-05-08T08:25:18.563 INFO:teuthology.orchestra.run.smithi031.stdout:733 trunc from 0xa9b4435 to 0x8fd2be8 2024-05-08T08:25:18.572 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:18.580+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:18.590 INFO:teuthology.orchestra.run.smithi031.stdout:735 read 0x13eccc3 thru 0x13ef0b6 (0x23f4 bytes) 2024-05-08T08:25:18.590 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.590 INFO:teuthology.orchestra.run.smithi031.stdout:738 read 0x21bfbf0 thru 0x21c448c (0x489d bytes) 2024-05-08T08:25:18.591 INFO:teuthology.orchestra.run.smithi031.stdout:739 write 0x68a05ac thru 0x68a8c81 (0x86d6 bytes) 2024-05-08T08:25:18.595 INFO:teuthology.orchestra.run.smithi031.stdout:740 writesame 0xf6b37 thru 0xfc49f data_size 0x1(0x5969 bytes) 2024-05-08T08:25:18.611 INFO:teuthology.orchestra.run.smithi031.stdout:741 writesame 0x1990bc2 thru 0x199fb25 data_size 0x6(0xef64 bytes) 2024-05-08T08:25:18.619 INFO:teuthology.orchestra.run.smithi031.stdout:742 read 0x7a51cc0 thru 0x7a56c29 (0x4f6a bytes) 2024-05-08T08:25:18.620 INFO:teuthology.orchestra.run.smithi031.stdout:743 write 0x548847c thru 0x5491947 (0x94cc bytes) 2024-05-08T08:25:18.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.626 INFO:teuthology.orchestra.run.smithi031.stdout:746 write 0x60fbfd3 thru 0x6107e92 (0xbec0 bytes) 2024-05-08T08:25:18.634 INFO:teuthology.orchestra.run.smithi031.stdout:748 read 0x3deb6a2 thru 0x3dfa7f0 (0xf14f bytes) 2024-05-08T08:25:18.635 INFO:teuthology.orchestra.run.smithi031.stdout:749 write 0x218995d thru 0x218b3dc (0x1a80 bytes) 2024-05-08T08:25:18.641 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.641 INFO:teuthology.orchestra.run.smithi031.stdout:751 write 0x180dda4 thru 0x180f173 (0x13d0 bytes) 2024-05-08T08:25:18.645 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.645 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.646 INFO:teuthology.orchestra.run.smithi031.stdout:755 punch from 0x8194b46 to 0x819821a, (0x36d4 bytes) 2024-05-08T08:25:18.646 INFO:teuthology.orchestra.run.smithi031.stdout:756 write 0x37aa8d7 thru 0x37b4ee2 (0xa60c bytes) 2024-05-08T08:25:18.650 INFO:teuthology.orchestra.run.smithi031.stdout:757 read 0x40c8e05 thru 0x40d4edc (0xc0d8 bytes) 2024-05-08T08:25:18.651 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.651 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.651 INFO:teuthology.orchestra.run.smithi031.stdout:760 writesame 0x6c5a8c2 thru 0x6c63048 data_size 0x2d(0x8787 bytes) 2024-05-08T08:25:18.663 INFO:teuthology.orchestra.run.smithi031.stdout:761 trunc from 0x8fd2be8 to 0xdf2378a 2024-05-08T08:25:18.669 INFO:teuthology.orchestra.run.smithi031.stdout:762 punch from 0xc8dd0be to 0xc8e96f2, (0xc634 bytes) 2024-05-08T08:25:18.669 INFO:teuthology.orchestra.run.smithi031.stdout:763 write 0xc5d29 thru 0xd3acd (0xdda5 bytes) 2024-05-08T08:25:18.673 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:18.679+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:18.682 INFO:teuthology.orchestra.run.smithi031.stdout:764 trunc from 0xdf2378a to 0x7b72a85 2024-05-08T08:25:18.685 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:18.685 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:18.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:18.704 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.704 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.716 INFO:teuthology.orchestra.run.smithi031.stdout:770 write 0xd047a17 thru 0xd050b46 (0x9130 bytes) 2024-05-08T08:25:18.720 INFO:teuthology.orchestra.run.smithi031.stdout:771 read 0x4a4c88a thru 0x4a56914 (0xa08b bytes) 2024-05-08T08:25:18.721 INFO:teuthology.orchestra.run.smithi031.stdout:772 writesame 0xad2d93e thru 0xad2fbe8 data_size 0x19(0x22ab bytes) 2024-05-08T08:25:18.723 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.724 INFO:teuthology.orchestra.run.smithi031.stdout:775 write 0x4803767 thru 0x4812726 (0xefc0 bytes) 2024-05-08T08:25:18.738 INFO:teuthology.orchestra.run.smithi031.stdout:776 writesame 0xe7af281 thru 0xe7af97f data_size 0x3(0x6ff bytes) 2024-05-08T08:25:18.738 INFO:teuthology.orchestra.run.smithi031.stdout:778 writesame 0x2593dc6 thru 0x2593fd0 data_size 0x1(0x20b bytes) 2024-05-08T08:25:18.742 INFO:teuthology.orchestra.run.smithi031.stdout:780 read 0x4cedc0b thru 0x4cfd311 (0xf707 bytes) 2024-05-08T08:25:18.743 INFO:teuthology.orchestra.run.smithi031.stdout:781 read 0x974c845 thru 0x97516b2 (0x4e6e bytes) 2024-05-08T08:25:18.743 INFO:teuthology.orchestra.run.smithi031.stdout:782 read 0xe37c115 thru 0xe38a401 (0xe2ed bytes) 2024-05-08T08:25:18.743 INFO:teuthology.orchestra.run.smithi031.stdout:783 punch from 0x9a15545 to 0x9a1aa6d, (0x5528 bytes) 2024-05-08T08:25:18.743 INFO:teuthology.orchestra.run.smithi031.stdout:784 writesame 0xace2125 thru 0xacf1a12 data_size 0x81(0xf8ee bytes) 2024-05-08T08:25:18.747 INFO:teuthology.orchestra.run.smithi031.stdout:787 punch from 0xaefec0a to 0xaf0dbbd, (0xefb3 bytes) 2024-05-08T08:25:18.747 INFO:teuthology.orchestra.run.smithi031.stdout:788 write 0xb32f6ce thru 0xb337f9c (0x88cf bytes) 2024-05-08T08:25:18.753 INFO:teuthology.orchestra.run.smithi031.stdout:790 write 0xd80e836 thru 0xd813266 (0x4a31 bytes) 2024-05-08T08:25:18.756 INFO:teuthology.orchestra.run.smithi031.stdout:791 write 0x489f556 thru 0x48ac9fe (0xd4a9 bytes) 2024-05-08T08:25:18.762 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.762 INFO:teuthology.orchestra.run.smithi031.stdout:793 write 0x79d442a thru 0x79e0c17 (0xc7ee bytes) 2024-05-08T08:25:18.769 INFO:teuthology.orchestra.run.smithi031.stdout:794 read 0xe41c730 thru 0xe41cb31 (0x402 bytes) 2024-05-08T08:25:18.774 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:18.780+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:18.775 INFO:teuthology.orchestra.run.smithi031.stdout:796 trunc from 0xe7af980 to 0x9cf40eb 2024-05-08T08:25:18.801 INFO:teuthology.orchestra.run.smithi031.stdout:798 write 0xea16ca8 thru 0xea1d493 (0x67ec bytes) 2024-05-08T08:25:18.805 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.805 INFO:teuthology.orchestra.run.smithi031.stdout:801 trunc from 0xea1d494 to 0xe7adbe7 2024-05-08T08:25:18.815 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.816 INFO:teuthology.orchestra.run.smithi031.stdout:803 write 0x54cfb3e thru 0x54d01e9 (0x6ac bytes) 2024-05-08T08:25:18.832 INFO:teuthology.orchestra.run.smithi031.stdout:805 trunc from 0xe7adbe7 to 0x7b9cadd 2024-05-08T08:25:18.839 INFO:teuthology.orchestra.run.smithi031.stdout:806 read 0x71dda20 thru 0x71e3f47 (0x6528 bytes) 2024-05-08T08:25:18.840 INFO:teuthology.orchestra.run.smithi031.stdout:807 punch from 0x4f954a4 to 0x4fa1f1f, (0xca7b bytes) 2024-05-08T08:25:18.840 INFO:teuthology.orchestra.run.smithi031.stdout:808 read 0x3a0467f thru 0x3a10cd9 (0xc65b bytes) 2024-05-08T08:25:18.842 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:18.842 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:18.853 INFO:teuthology.orchestra.run.smithi031.stdout:810 write 0xcdcd4cd thru 0xcdd26ed (0x5221 bytes) 2024-05-08T08:25:18.857 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:18.859 INFO:teuthology.orchestra.run.smithi031.stdout:811 writesame 0x8734e67 thru 0x873f3df data_size 0x1(0xa579 bytes) 2024-05-08T08:25:18.874 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:18.881+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:18.878 INFO:teuthology.orchestra.run.smithi031.stdout:812 write 0xa866782 thru 0xa8764ea (0xfd69 bytes) 2024-05-08T08:25:18.887 INFO:teuthology.orchestra.run.smithi031.stdout:813 writesame 0x596751e thru 0x596a27f data_size 0x25(0x2d62 bytes) 2024-05-08T08:25:18.892 INFO:teuthology.orchestra.run.smithi031.stdout:814 read 0x6a759eb thru 0x6a7caa6 (0x70bc bytes) 2024-05-08T08:25:18.892 INFO:teuthology.orchestra.run.smithi031.stdout:815 punch from 0x261e833 to 0x2620f24, (0x26f1 bytes) 2024-05-08T08:25:18.893 INFO:teuthology.orchestra.run.smithi031.stdout:816 read 0x53891c1 thru 0x539761c (0xe45c bytes) 2024-05-08T08:25:18.893 INFO:teuthology.orchestra.run.smithi031.stdout:817 write 0xad22aaf thru 0xad305c5 (0xdb17 bytes) 2024-05-08T08:25:18.898 INFO:teuthology.orchestra.run.smithi031.stdout:818 read 0x3ef208f thru 0x3ef7cf7 (0x5c69 bytes) 2024-05-08T08:25:18.898 INFO:teuthology.orchestra.run.smithi031.stdout:819 punch from 0xa13580a to 0xa13c47a, (0x6c70 bytes) 2024-05-08T08:25:18.898 INFO:teuthology.orchestra.run.smithi031.stdout:820 trunc from 0xcdd26ee to 0xd1284c4 2024-05-08T08:25:18.903 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:18.903 INFO:teuthology.orchestra.run.smithi031.stdout:822 write 0xd0abcd6 thru 0xd0b7519 (0xb844 bytes) 2024-05-08T08:25:18.906 INFO:teuthology.orchestra.run.smithi031.stdout:824 write 0x4c7eeca thru 0x4c8e8b0 (0xf9e7 bytes) 2024-05-08T08:25:18.925 INFO:teuthology.orchestra.run.smithi031.stdout:825 trunc from 0xd1284c4 to 0x2919eed 2024-05-08T08:25:18.954 INFO:teuthology.orchestra.run.smithi031.stdout:826 read 0x9ff527 thru 0xa077af (0x8289 bytes) 2024-05-08T08:25:18.955 INFO:teuthology.orchestra.run.smithi031.stdout:828 writesame 0x11b0ec thru 0x11efb4 data_size 0x1(0x3ec9 bytes) 2024-05-08T08:25:18.973 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T08:25:18.974 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:25:18.974 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T08:25:18.974 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T08:25:18.974 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:25:18.975 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:18.982+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:18.976 INFO:teuthology.orchestra.run.smithi031.stdout:829 trunc from 0x2919eed to 0xb873c84 2024-05-08T08:25:18.979 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:25:18.987 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:18.987 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:18.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:19.014 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T08:25:19.014 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.1 2024-05-08T08:25:19.014 INFO:tasks.thrashosds.thrasher:remote for osd 1 is ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:25:19.014 DEBUG:teuthology.orchestra.run.smithi031:> type ceph-objectstore-tool 2024-05-08T08:25:19.033 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:19.067 INFO:teuthology.orchestra.run.smithi031.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T08:25:19.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2024-05-08T08:25:19.076 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:19.083+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:19.163 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:19.164 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:19.175 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:19.176 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:19.184+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:19.191 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:19.191 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:19.277 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:19.284+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:19.282 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:19.282 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:19.291 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:19.378 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:19.386+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:19.436 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:19.437 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:19.446 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:19.489 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:19.496+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:19.552 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:19.552 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:19.561 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:19.580 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:19.588+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:19.681 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:19.688+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:19.707 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:19.707 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:19.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:19.783 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:19.790+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:19.862 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:19.862 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:19.874 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:19.918 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:19.925+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:19.983 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:19.983 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:19.984 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:19.991+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:19.994 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:20.086 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:20.093+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:20.140 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:20.140 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:20.148 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:20.192 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:20.199+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:20.256 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:20.256 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:3.6 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:3.8 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:3.4 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:3.1a 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:3.19 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:3.0 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:2.0 2024-05-08T08:25:20.261 INFO:teuthology.orchestra.run.smithi031.stdout:1.3 2024-05-08T08:25:20.265 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:20.305 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:20.312+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:20.388 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:20.395+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:20.412 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:20.412 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:20.421 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:20.505 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:20.512+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:20.568 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:20.568 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:20.577 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:20.591 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:20.598+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:20.691 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:20.698+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:20.723 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:20.723 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:20.732 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:20.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 3.1a --file /var/log/ceph/exp.3.1a.1 2024-05-08T08:25:20.816 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:20.823+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:20.878 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:20.879 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:20.887 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:20.928 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:20.935+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:20.993 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:25:20.994 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:21.002+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:21.003 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:21.003 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:21.003 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:21.004 INFO:teuthology.orchestra.run.smithi031.stdout:831 trunc from 0xb873c84 to 0x4c492a5 2024-05-08T08:25:21.012 INFO:teuthology.orchestra.run.smithi031.stdout:833 read 0x3474755 thru 0x347ca74 (0x8320 bytes) 2024-05-08T08:25:21.012 INFO:teuthology.orchestra.run.smithi031.stdout:835 punch from 0xc3da02 to 0xc3f245, (0x1843 bytes) 2024-05-08T08:25:21.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:21.026 INFO:teuthology.orchestra.run.smithi031.stdout:836 write 0xc6d5d1e thru 0xc6e4b03 (0xede6 bytes) 2024-05-08T08:25:21.032 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.032 INFO:teuthology.orchestra.run.smithi031.stdout:839 punch from 0xdfd622 to 0xe08acc, (0xb4aa bytes) 2024-05-08T08:25:21.032 INFO:teuthology.orchestra.run.smithi031.stdout:840 read 0x3a1e060 thru 0x3a265d1 (0x8572 bytes) 2024-05-08T08:25:21.038 INFO:teuthology.orchestra.run.smithi031.stdout:842 write 0xcc37e82 thru 0xcc472c0 (0xf43f bytes) 2024-05-08T08:25:21.044 INFO:teuthology.orchestra.run.smithi031.stdout:844 punch from 0x756d807 to 0x756fd0a, (0x2503 bytes) 2024-05-08T08:25:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:845 read 0x5eb688f thru 0x5eb83b4 (0x1b26 bytes) 2024-05-08T08:25:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:852 punch from 0x85e474a to 0x85eb7ca, (0x7080 bytes) 2024-05-08T08:25:21.045 INFO:teuthology.orchestra.run.smithi031.stdout:855 write 0x2a74b79 thru 0x2a75f0e (0x1396 bytes) 2024-05-08T08:25:21.047 INFO:teuthology.orchestra.run.smithi031.stdout:856 write 0x3c99108 thru 0x3c9d2b9 (0x41b2 bytes) 2024-05-08T08:25:21.051 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.051 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.051 INFO:teuthology.orchestra.run.smithi031.stdout:861 write 0xc7eec87 thru 0xc7fbf21 (0xd29b bytes) 2024-05-08T08:25:21.055 INFO:teuthology.orchestra.run.smithi031.stdout:864 writesame 0x4ab2ab9 thru 0x4ab9f72 data_size 0x2(0x74ba bytes) 2024-05-08T08:25:21.064 INFO:teuthology.orchestra.run.smithi031.stdout:870 write 0x41854eb thru 0x418bf21 (0x6a37 bytes) 2024-05-08T08:25:21.069 INFO:teuthology.orchestra.run.smithi031.stdout:871 write 0x4f090d0 thru 0x4f0cbaf (0x3ae0 bytes) 2024-05-08T08:25:21.072 INFO:teuthology.orchestra.run.smithi031.stdout:874 write 0x79b6cf6 thru 0x79c26f0 (0xb9fb bytes) 2024-05-08T08:25:21.077 INFO:teuthology.orchestra.run.smithi031.stdout:876 punch from 0xacbd1cc to 0xacc6e63, (0x9c97 bytes) 2024-05-08T08:25:21.077 INFO:teuthology.orchestra.run.smithi031.stdout:878 writesame 0x70dd3f7 thru 0x70df3f6 data_size 0x20(0x2000 bytes) 2024-05-08T08:25:21.081 INFO:teuthology.orchestra.run.smithi031.stdout:880 read 0x1005da6 thru 0x1006d25 (0xf80 bytes) 2024-05-08T08:25:21.082 INFO:teuthology.orchestra.run.smithi031.stdout:882 write 0x4daa08f thru 0x4db3708 (0x967a bytes) 2024-05-08T08:25:21.086 INFO:teuthology.orchestra.run.smithi031.stdout:884 read 0x1a1a701 thru 0x1a2426e (0x9b6e bytes) 2024-05-08T08:25:21.092 INFO:teuthology.orchestra.run.smithi031.stdout:886 trunc from 0xcc472c1 to 0x7a0613c 2024-05-08T08:25:21.096 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:21.102+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:887 punch from 0x397df42 to 0x397eb4a, (0xc08 bytes) 2024-05-08T08:25:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:889 read 0x5bd13bb thru 0x5bd49f0 (0x3636 bytes) 2024-05-08T08:25:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:890 punch from 0x23fe6b3 to 0x240b3a8, (0xccf5 bytes) 2024-05-08T08:25:21.107 INFO:teuthology.orchestra.run.smithi031.stdout:891 writesame 0x6962f58 thru 0x696b524 data_size 0x1(0x85cd bytes) 2024-05-08T08:25:21.125 INFO:teuthology.orchestra.run.smithi031.stdout:893 trunc from 0x7a0613c to 0x2af0843 2024-05-08T08:25:21.148 INFO:teuthology.orchestra.run.smithi031.stdout:894 write 0x36207dc thru 0x362d72b (0xcf50 bytes) 2024-05-08T08:25:21.164 INFO:teuthology.orchestra.run.smithi031.stdout:895 write 0x829a9d1 thru 0x829eb22 (0x4152 bytes) 2024-05-08T08:25:21.165 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:21.165 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:21.166 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:21.166 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:21.173 INFO:teuthology.orchestra.run.smithi031.stdout:896 trunc from 0x829eb23 to 0x4c9e0e9 2024-05-08T08:25:21.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:21.183 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.192 INFO:teuthology.orchestra.run.smithi031.stdout:898 trunc from 0x4c9e0e9 to 0xc54d719 2024-05-08T08:25:21.196 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:21.203+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:21.200 INFO:teuthology.orchestra.run.smithi031.stdout:899 trunc from 0xc54d719 to 0xdb7f37a 2024-05-08T08:25:21.207 INFO:teuthology.orchestra.run.smithi031.stdout:900 punch from 0x73000c9 to 0x7309702, (0x9639 bytes) 2024-05-08T08:25:21.207 INFO:teuthology.orchestra.run.smithi031.stdout:901 read 0x302c74f thru 0x302cd7e (0x630 bytes) 2024-05-08T08:25:21.207 INFO:teuthology.orchestra.run.smithi031.stdout:903 write 0x4fc4dca thru 0x4fcb9b2 (0x6be9 bytes) 2024-05-08T08:25:21.213 INFO:teuthology.orchestra.run.smithi031.stdout:905 punch from 0x662f2ae to 0x6635583, (0x62d5 bytes) 2024-05-08T08:25:21.214 INFO:teuthology.orchestra.run.smithi031.stdout:906 writesame 0xb52d73b thru 0xb532aef data_size 0x3(0x53b5 bytes) 2024-05-08T08:25:21.221 INFO:teuthology.orchestra.run.smithi031.stdout:908 writesame 0x74b021a thru 0x74bf079 data_size 0x10(0xee60 bytes) 2024-05-08T08:25:21.226 INFO:teuthology.orchestra.run.smithi031.stdout:909 write 0x127741b thru 0x127a76c (0x3352 bytes) 2024-05-08T08:25:21.232 INFO:teuthology.orchestra.run.smithi031.stdout:910 writesame 0xce13166 thru 0xce1a000 data_size 0x7(0x6e9b bytes) 2024-05-08T08:25:21.238 INFO:teuthology.orchestra.run.smithi031.stdout:911 writesame 0x5d5e3dd thru 0x5d65827 data_size 0x1(0x744b bytes) 2024-05-08T08:25:21.252 INFO:teuthology.orchestra.run.smithi031.stdout:913 write 0x8b55e44 thru 0x8b638b5 (0xda72 bytes) 2024-05-08T08:25:21.267 INFO:teuthology.orchestra.run.smithi031.stdout:914 trunc from 0xdb7f37a to 0x5c955eb 2024-05-08T08:25:21.293 INFO:teuthology.orchestra.run.smithi031.stdout:915 writesame 0xe0a27fb thru 0xe0ae1eb data_size 0x2b(0xb9f1 bytes) 2024-05-08T08:25:21.293 INFO:teuthology.orchestra.run.smithi031.stdout:917 read 0x8e27670 thru 0x8e2fb1a (0x84ab bytes) 2024-05-08T08:25:21.297 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:21.304+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:21.305 INFO:teuthology.orchestra.run.smithi031.stdout:918 trunc from 0xe0ae1ec to 0x2ba4992 2024-05-08T08:25:21.317 INFO:teuthology.orchestra.run.smithi031.stdout:919 read 0xb45d08 thru 0xb52134 (0xc42d bytes) 2024-05-08T08:25:21.318 INFO:teuthology.orchestra.run.smithi031.stdout:920 read 0x258ba00 thru 0x259280f (0x6e10 bytes) 2024-05-08T08:25:21.318 INFO:teuthology.orchestra.run.smithi031.stdout:923 read 0xc7d578 thru 0xc81466 (0x3eef bytes) 2024-05-08T08:25:21.319 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.319 INFO:teuthology.orchestra.run.smithi031.stdout:926 read 0x1d2e8c1 thru 0x1d2f925 (0x1065 bytes) 2024-05-08T08:25:21.326 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6559 3.1b 3:dd77e507:::rbd_data.119152d8614e.0000000000000010:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.913627+0000", 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.419041504, 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014663238, 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6559 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913627+0000", 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913627+0000", 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913628+0000", 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1000000000000001e-06 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913634+0000", 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.648e-06 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913647+0000", 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2696e-05 2024-05-08T08:25:21.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913654+0000", 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1659999999999997e-06 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913700+0000", 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6054999999999999e-05 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.913746+0000", 2024-05-08T08:25:21.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6489999999999997e-05 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.916214+0000", 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002468217 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.928114+0000", 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01189935 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:21.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.928169+0000", 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4954999999999998e-05 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.928235+0000", 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5876000000000004e-05 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.928290+0000", 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5685e-05 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.332 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6562 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4013996200000003, 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024048491000000002, 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.333 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6562 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931270+0000", 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.19e-06 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931276+0000", 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4790000000000004e-06 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931286+0000", 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.044e-05 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931292+0000", 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931346+0000", 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4274999999999998e-05 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931398+0000", 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1999999999999997e-05 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.935504+0000", 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041055170000000004 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955221+0000", 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019717443000000001 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955234+0000", 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2191999999999999e-05 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955271+0000", 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7008000000000002e-05 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955317+0000", 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6796999999999999e-05 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6984 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 5505024~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.186574+0000", 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1460942779999996, 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014984279, 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6984 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186574+0000", 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186574+0000", 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186575+0000", 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:25:21.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186577+0000", 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5299999999999999e-06 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186584+0000", 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.489e-06 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186589+0000", 2024-05-08T08:25:21.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.200896+0000", 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014306874000000001 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.200979+0000", 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2579999999999999e-05 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.201559+0000", 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058007999999999996 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6985 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 8126464~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.186595+0000", 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1460734600000002, 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015812522999999998, 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6985 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186595+0000", 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186595+0000", 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186595+0000", 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.15e-07 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186598+0000", 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.216e-06 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186604+0000", 2024-05-08T08:25:21.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.186608+0000", 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7600000000000002e-06 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.201611+0000", 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015002587 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.201704+0000", 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2789999999999998e-05 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.202408+0000", 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00070384199999999999 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7002 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 1835008~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188146+0000", 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1445230520000003, 2024-05-08T08:25:21.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014983724, 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7002 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188146+0000", 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188146+0000", 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188147+0000", 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9999999999999995e-07 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188152+0000", 2024-05-08T08:25:21.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1680000000000003e-06 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188168+0000", 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6356999999999999e-05 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188176+0000", 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4430000000000003e-06 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.202445+0000", 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014268332999999999 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.202542+0000", 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6872000000000004e-05 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203129+0000", 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058755099999999996 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7003 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4456448~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188191+0000", 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1444776880000003, 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015719848000000002, 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7003 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188191+0000", 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188191+0000", 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188191+0000", 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1600000000000001e-07 2024-05-08T08:25:21.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188194+0000", 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188204+0000", 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4409999999999999e-06 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188210+0000", 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7880000000000003e-06 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203179+0000", 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.014968917999999999 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203265+0000", 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6091e-05 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203911+0000", 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00064614000000000002 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7004 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7077888~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1444497719999998, 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016625803000000001, 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7004 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:21.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188222+0000", 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7219999999999999e-06 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188229+0000", 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6359999999999997e-06 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188234+0000", 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.502e-06 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203959+0000", 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015725330999999999 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204038+0000", 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8379000000000003e-05 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204845+0000", 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080675500000000004 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7020 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1443605730000002, 2024-05-08T08:25:21.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017204636999999998, 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7020 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188311+0000", 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-05-08T08:25:21.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188317+0000", 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7959999999999998e-06 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188322+0000", 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204639+0000", 2024-05-08T08:25:21.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016316714 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204730+0000", 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0771000000000002e-05 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205513+0000", 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078277700000000002 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7021 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3145728~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.144339596, 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017850441000000002, 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7021 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6199999999999999e-07 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188331+0000", 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1069999999999999e-06 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188338+0000", 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188343+0000", 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205565+0000", 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017221841000000002 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205665+0000", 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9530000000000004e-05 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206179+0000", 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051477999999999995 2024-05-08T08:25:21.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7022 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 5767168~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1443184759999996, 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018456931999999999, 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7022 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188352+0000", 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188358+0000", 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-05-08T08:25:21.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188363+0000", 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206229+0000", 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017865941 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206327+0000", 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7888999999999997e-05 2024-05-08T08:25:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206807+0000", 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048019000000000002 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7038 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1442125719999998, 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019193815999999999, 2024-05-08T08:25:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7038 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188457+0000", 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188461+0000", 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188469+0000", 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188477+0000", 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3559999999999993e-06 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206829+0000", 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018352456 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206928+0000", 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8981999999999995e-05 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207650+0000", 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072148600000000002 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7039 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1441798309999998, 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491153000000001, 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7039 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188492+0000", 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188499+0000", 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7469999999999999e-06 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188505+0000", 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207495+0000", 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990683000000001 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207577+0000", 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.174e-05 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207980+0000", 2024-05-08T08:25:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040296899999999998 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7040 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1441565420000002, 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020064733000000001, 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7040 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188516+0000", 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999999e-06 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188522+0000", 2024-05-08T08:25:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188527+0000", 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208019+0000", 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491871000000001 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208118+0000", 2024-05-08T08:25:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8855000000000001e-05 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208577+0000", 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000458784 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7056 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1440757189999999, 2024-05-08T08:25:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020583127999999999, 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188595+0000", 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0679999999999999e-06 2024-05-08T08:25:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188603+0000", 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999997e-06 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188609+0000", 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208623+0000", 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0200141 2024-05-08T08:25:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208713+0000", 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9806e-05 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209176+0000", 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046324300000000001 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7057 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4718592~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1440226019999997, 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021149303000000001, 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188647+0000", 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:25:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188650+0000", 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7100000000000001e-06 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188657+0000", 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188665+0000", 2024-05-08T08:25:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4410000000000004e-06 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209211+0000", 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020546426999999999 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209299+0000", 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8139000000000004e-05 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209795+0000", 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049584800000000001 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7058 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7340032~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1439922889999998, 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021706842, 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7058 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188677+0000", 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188681+0000", 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188688+0000", 2024-05-08T08:25:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.494e-06 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188693+0000", 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209838+0000", 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021144677000000001 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209901+0000", 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3322000000000001e-05 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210383+0000", 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048203800000000001 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7073 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2359296~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1438967800000004, 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022130316000000001, 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188775+0000", 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188782+0000", 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8630000000000008e-06 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188787+0000", 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210421+0000", 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021633454999999999 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210481+0000", 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9911000000000003e-05 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210902+0000", 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042136000000000002 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7074 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4980736~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1438744840000004, 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022761245999999999, 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-05-08T08:25:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188795+0000", 2024-05-08T08:25:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188797+0000", 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188803+0000", 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188807+0000", 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210951+0000", 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022143843 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211043+0000", 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2044000000000002e-05 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211555+0000", 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051209900000000002 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7075 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7602176~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.143854557, 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025123269, 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7075 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188817+0000", 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188823+0000", 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188828+0000", 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211585+0000", 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022757553999999999 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211640+0000", 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000002e-05 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213937+0000", 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002297182 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7090 3.12 3:4c3c5acc:::rbd_data.119120fdbc10.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1572864~19660800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.1343804329999996, 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017096892999999998, 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7090 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198290+0000", 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.666e-06 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198297+0000", 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198308+0000", 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0641e-05 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198314+0000", 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7850000000000003e-06 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213971+0000", 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015656415 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214141+0000", 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016982999999999999 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214349+0000", 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020874199999999999 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215032+0000", 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068226599999999995 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215223+0000", 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191263 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215286+0000", 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000004e-05 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215385+0000", 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9161999999999997e-05 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:21.418 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:21.419 INFO:teuthology.orchestra.run.smithi031.stdout:927 writesame 0x48ed62d thru 0x48f5e3e data_size 0x1(0x8812 bytes) 2024-05-08T08:25:21.419 INFO:teuthology.orchestra.run.smithi031.stdout:929 punch from 0xe06155 to 0xe07486, (0x1331 bytes) 2024-05-08T08:25:21.420 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:21.421 INFO:teuthology.orchestra.run.smithi031.stdout:931 writesame 0x9e0eea6 thru 0x9e1226c data_size 0xb(0x33c7 bytes) 2024-05-08T08:25:21.421 INFO:teuthology.orchestra.run.smithi031.stdout:932 writesame 0x2d08f5d thru 0x2d0c130 data_size 0x6(0x31d4 bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:934 punch from 0x11844ed to 0x11921fc, (0xdd0f bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:935 punch from 0x137b1 to 0x1da67, (0xa2b6 bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:936 write 0x299c65e thru 0x29a1f12 (0x58b5 bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:937 writesame 0xc31bb53 thru 0xc325172 data_size 0x10(0x9620 bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:939 trunc from 0xc325173 to 0x2ad6d60 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:940 punch from 0x24af999 to 0x24b1c0b, (0x2272 bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:941 write 0xe7277b9 thru 0xe733775 (0xbfbd bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:943 punch from 0xa2f3c47 to 0xa2fb9b1, (0x7d6a bytes) 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:944 trunc from 0xe733776 to 0x55e9e50 2024-05-08T08:25:21.422 INFO:teuthology.orchestra.run.smithi031.stdout:946 write 0x2c0c793 thru 0x2c1b938 (0xf1a6 bytes) 2024-05-08T08:25:21.423 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:21.426+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:21.426 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.437 INFO:teuthology.orchestra.run.smithi031.stdout:949 writesame 0x9f27b86 thru 0x9f31749 data_size 0x6(0x9bc4 bytes) 2024-05-08T08:25:21.441 INFO:teuthology.orchestra.run.smithi031.stdout:950 trunc from 0x9f3174a to 0xdb99cf5 2024-05-08T08:25:21.446 INFO:teuthology.orchestra.run.smithi031.stdout:953 punch from 0x6895dd0 to 0x689744e, (0x167e bytes) 2024-05-08T08:25:21.447 INFO:teuthology.orchestra.run.smithi031.stdout:954 writesame 0xb3b05c3 thru 0xb3b9bd0 data_size 0x1(0x960e bytes) 2024-05-08T08:25:21.463 INFO:teuthology.orchestra.run.smithi031.stdout:955 writesame 0x910ee57 thru 0x911b45f data_size 0x9(0xc609 bytes) 2024-05-08T08:25:21.468 INFO:teuthology.orchestra.run.smithi031.stdout:957 writesame 0x15597b1 thru 0x155a8e8 data_size 0x13(0x1138 bytes) 2024-05-08T08:25:21.473 INFO:teuthology.orchestra.run.smithi031.stdout:959 read 0x3f7fcbc thru 0x3f8ba1e (0xbd63 bytes) 2024-05-08T08:25:21.473 INFO:teuthology.orchestra.run.smithi031.stdout:960 writesame 0x88346d6 thru 0x8840cb6 data_size 0x1(0xc5e1 bytes) 2024-05-08T08:25:21.491 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:21.505 INFO:teuthology.orchestra.run.smithi031.stdout:962 trunc from 0xdb99cf5 to 0x121fa65 2024-05-08T08:25:21.519 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:21.526+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:21.533 INFO:teuthology.orchestra.run.smithi031.stdout:963 write 0x4841d9b thru 0x484c019 (0xa27f bytes) 2024-05-08T08:25:21.537 INFO:teuthology.orchestra.run.smithi031.stdout:964 clone 8 order 19 su 131072 sc 7 2024-05-08T08:25:21.553 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:21.553 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:21.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:21.646 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:21.654+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:21.713 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:21.714 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:21.721 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:21.727+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:21.722 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:21.821 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:21.829+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:21.871 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:21.871 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:21.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:21.924 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:21.931+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:21.925 INFO:teuthology.orchestra.run.smithi031.stderr:Exporting 3.1a info 3.1a( v 29'71 (0'0,29'71] local-lis/les=18/19 n=12 ec=18/18 lis/c=18/18 les/c/f=19/19/0 sis=18) 2024-05-08T08:25:21.931 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:581f9ee9:::rbd_data.1191f7f15880.000000000000000d:head# 2024-05-08T08:25:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:588df1ca:::rbd_data.1191f7f15880.0000000000000030:head# 2024-05-08T08:25:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:59a2de6e:::rbd_data.1191f7f15880.0000000000000053:head# 2024-05-08T08:25:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5aec0c35:::rbd_object_map.119120fdbc10:head# 2024-05-08T08:25:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5b7a5ee6:::rbd_data.1191f7f15880.000000000000003e:head# 2024-05-08T08:25:21.932 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head# 2024-05-08T08:25:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5cae588f:::rbd_data.1191f7f15880.000000000000000b:head# 2024-05-08T08:25:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:4# 2024-05-08T08:25:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head# 2024-05-08T08:25:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5cea7035:::rbd_info:head# 2024-05-08T08:25:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5d02ff65:::rbd_data.1191f7f15880.0000000000000061:head# 2024-05-08T08:25:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:Read #3:5d86de77:::rbd_data.1191f7f15880.0000000000000046:head# 2024-05-08T08:25:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:Export successful 2024-05-08T08:25:21.994 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:25:21.996 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:21.996 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:22.023 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:22.030+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:22.101 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone7 intact 2024-05-08T08:25:22.124 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:22.131+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:22.165 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:22.165 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.175 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:22.225 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:22.232+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:22.262 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-05-08T08:25:22.295 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:22.295 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.307 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:22.351 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:22.358+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:22.379 INFO:teuthology.orchestra.run.smithi031.stdout:965 trunc from 0x484c01a to 0xb03ba4a 2024-05-08T08:25:22.388 INFO:teuthology.orchestra.run.smithi031.stdout:966 writesame 0x8f24372 thru 0x8f312ef data_size 0x75(0xcf7e bytes) 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:967 read 0x69bc943 thru 0x69bd3f2 (0xab0 bytes) 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:968 read 0x2d37ddb thru 0x2d3eb2a (0x6d50 bytes) 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:969 read 0x400143a thru 0x4008926 (0x74ed bytes) 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:970 read 0xabc8303 thru 0xabcd54e (0x524c bytes) 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:971 read 0x181c994 thru 0x182a2ef (0xd95c bytes) 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:975 read 0x7bd7d78 thru 0x7bd9bbe (0x1e47 bytes) 2024-05-08T08:25:22.393 INFO:teuthology.orchestra.run.smithi031.stdout:976 write 0x8e78a59 thru 0x8e859fc (0xcfa4 bytes) 2024-05-08T08:25:22.397 INFO:teuthology.orchestra.run.smithi031.stdout:977 write 0x239aeb7 thru 0x239cfe5 (0x212f bytes) 2024-05-08T08:25:22.399 INFO:teuthology.orchestra.run.smithi031.stdout:980 writesame 0x51d459e thru 0x51e3e43 data_size 0x67(0xf8a6 bytes) 2024-05-08T08:25:22.404 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.404 INFO:teuthology.orchestra.run.smithi031.stdout:982 write 0x5877a0c thru 0x587a751 (0x2d46 bytes) 2024-05-08T08:25:22.407 INFO:teuthology.orchestra.run.smithi031.stdout:983 write 0x7dae52c thru 0x7db64fc (0x7fd1 bytes) 2024-05-08T08:25:22.411 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.411 INFO:teuthology.orchestra.run.smithi031.stdout:985 write 0x875cf34 thru 0x876b8a9 (0xe976 bytes) 2024-05-08T08:25:22.416 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:22.416 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.418 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.424 INFO:teuthology.orchestra.run.smithi031.stdout:987 trunc from 0xb03ba4a to 0x528e817 2024-05-08T08:25:22.426 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:22.426 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:22.432+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:22.436 INFO:teuthology.orchestra.run.smithi031.stdout:988 read 0x376c0ad thru 0x376c7de (0x732 bytes) 2024-05-08T08:25:22.436 INFO:teuthology.orchestra.run.smithi031.stdout:989 read 0x37852af thru 0x378d03c (0x7d8e bytes) 2024-05-08T08:25:22.437 INFO:teuthology.orchestra.run.smithi031.stdout:990 read 0x28a3c4 thru 0x28aa92 (0x6cf bytes) 2024-05-08T08:25:22.440 INFO:teuthology.orchestra.run.smithi031.stdout:991 trunc from 0x528e817 to 0x899442f 2024-05-08T08:25:22.446 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.446 INFO:teuthology.orchestra.run.smithi031.stdout:995 write 0x275fec6 thru 0x2760a50 (0xb8b bytes) 2024-05-08T08:25:22.450 INFO:teuthology.orchestra.run.smithi031.stdout:998 writesame 0x203c831 thru 0x2043fc9 data_size 0x1(0x7799 bytes) 2024-05-08T08:25:22.462 INFO:teuthology.orchestra.run.smithi031.stdout:999 write 0x851ed9 thru 0x860021 (0xe149 bytes) 2024-05-08T08:25:22.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 3.1a 2024-05-08T08:25:22.470 INFO:teuthology.orchestra.run.smithi031.stdout:1001 read 0x476618a thru 0x47724f2 (0xc369 bytes) 2024-05-08T08:25:22.479 INFO:teuthology.orchestra.run.smithi031.stdout:1003 write 0xbe71a2a thru 0xbe7b521 (0x9af8 bytes) 2024-05-08T08:25:22.482 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.482 INFO:teuthology.orchestra.run.smithi031.stdout:1005 read 0x4f1e388 thru 0x4f2957e (0xb1f7 bytes) 2024-05-08T08:25:22.482 INFO:teuthology.orchestra.run.smithi031.stdout:1006 punch from 0x8d98e09 to 0x8da0813, (0x7a0a bytes) 2024-05-08T08:25:22.482 INFO:teuthology.orchestra.run.smithi031.stdout:1007 write 0x1b25286 thru 0x1b347d3 (0xf54e bytes) 2024-05-08T08:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout:1008 read 0xa66b2e4 thru 0xa67931d (0xe03a bytes) 2024-05-08T08:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout:1009 read 0x55be70c thru 0x55ccbd1 (0xe4c6 bytes) 2024-05-08T08:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout:1012 read 0x7bb2d8a thru 0x7bb3234 (0x4ab bytes) 2024-05-08T08:25:22.487 INFO:teuthology.orchestra.run.smithi031.stdout:1013 write 0x1fd7c65 thru 0x1fe13fd (0x9799 bytes) 2024-05-08T08:25:22.493 INFO:teuthology.orchestra.run.smithi031.stdout:1014 writesame 0x99b8359 thru 0x99c169e data_size 0x7(0x9346 bytes) 2024-05-08T08:25:22.498 INFO:teuthology.orchestra.run.smithi031.stdout:1015 punch from 0x2549949 to 0x25526a3, (0x8d5a bytes) 2024-05-08T08:25:22.498 INFO:teuthology.orchestra.run.smithi031.stdout:1016 write 0xa8e42b7 thru 0xa8f3e0e (0xfb58 bytes) 2024-05-08T08:25:22.502 INFO:teuthology.orchestra.run.smithi031.stdout:1017 read 0x494ab9b thru 0x495315b (0x85c1 bytes) 2024-05-08T08:25:22.502 INFO:teuthology.orchestra.run.smithi031.stdout:1020 writesame 0x62b5d2f thru 0x62baa9f data_size 0x19(0x4d71 bytes) 2024-05-08T08:25:22.504 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.504 INFO:teuthology.orchestra.run.smithi031.stdout:1022 read 0xacccb6 thru 0xadc977 (0xfcc2 bytes) 2024-05-08T08:25:22.505 INFO:teuthology.orchestra.run.smithi031.stdout:1024 write 0xf0156a thru 0xf0ddba (0xc851 bytes) 2024-05-08T08:25:22.516 INFO:teuthology.orchestra.run.smithi031.stdout:1026 write 0xdc40689 thru 0xdc40b3a (0x4b2 bytes) 2024-05-08T08:25:22.519 INFO:teuthology.orchestra.run.smithi031.stdout:1027 read 0x8773064 thru 0x8777540 (0x44dd bytes) 2024-05-08T08:25:22.520 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.527 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:22.534+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:22.533 INFO:teuthology.orchestra.run.smithi031.stdout:1029 trunc from 0xdc40b3b to 0x1ef4959 2024-05-08T08:25:22.562 INFO:teuthology.orchestra.run.smithi031.stdout:1030 write 0x550a278 thru 0x55155bc (0xb345 bytes) 2024-05-08T08:25:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:1031 punch from 0x486dd4 to 0x496612, (0xf83e bytes) 2024-05-08T08:25:22.565 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.572 INFO:teuthology.orchestra.run.smithi031.stdout:1033 writesame 0x5f415b3 thru 0x5f462c5 data_size 0x1(0x4d13 bytes) 2024-05-08T08:25:22.572 INFO:teuthology.orchestra.run.smithi031.stdout:1034 write 0x29b6b16 thru 0x29ba775 (0x3c60 bytes) 2024-05-08T08:25:22.575 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.575 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.575 INFO:teuthology.orchestra.run.smithi031.stdout:1040 punch from 0x213513b to 0x213a05d, (0x4f22 bytes) 2024-05-08T08:25:22.579 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:22.579 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.585 INFO:teuthology.orchestra.run.smithi031.stdout:1041 write 0xa932b7f thru 0xa9403a1 (0xd823 bytes) 2024-05-08T08:25:22.589 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:22.590 INFO:teuthology.orchestra.run.smithi031.stdout:1042 write 0x2e01bab thru 0x2e0e5a0 (0xc9f6 bytes) 2024-05-08T08:25:22.592 INFO:teuthology.orchestra.run.smithi031.stdout:1043 punch from 0x3526340 to 0x35313a3, (0xb063 bytes) 2024-05-08T08:25:22.600 INFO:teuthology.orchestra.run.smithi031.stdout:1044 write 0xd47f6e6 thru 0xd487cac (0x85c7 bytes) 2024-05-08T08:25:22.604 INFO:teuthology.orchestra.run.smithi031.stdout:1045 read 0x3b966c7 thru 0x3b9eec1 (0x87fb bytes) 2024-05-08T08:25:22.604 INFO:teuthology.orchestra.run.smithi031.stdout:1046 writesame 0xcdb40c6 thru 0xcdb95e8 data_size 0x5(0x5523 bytes) 2024-05-08T08:25:22.608 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.608 INFO:teuthology.orchestra.run.smithi031.stdout:1049 read 0x9983c32 thru 0x99869f3 (0x2dc2 bytes) 2024-05-08T08:25:22.611 INFO:teuthology.orchestra.run.smithi031.stdout:1050 trunc from 0xd487cad to 0xaa7b088 2024-05-08T08:25:22.623 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xeb026a0 2024-05-08T08:25:22.628 INFO:teuthology.orchestra.run.smithi031.stdout:1052 trunc from 0xaa7b088 to 0xeb026a0 2024-05-08T08:25:22.633 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:22.640+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:22.634 INFO:teuthology.orchestra.run.smithi031.stdout:1054 writesame 0xba731be thru 0xba74138 data_size 0x1(0xf7b bytes) 2024-05-08T08:25:22.637 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.637 INFO:teuthology.orchestra.run.smithi031.stdout:1056 read 0x7215ce0 thru 0x721a429 (0x474a bytes) 2024-05-08T08:25:22.637 INFO:teuthology.orchestra.run.smithi031.stdout:1057 write 0x1205331 thru 0x120c986 (0x7656 bytes) 2024-05-08T08:25:22.641 INFO:teuthology.orchestra.run.smithi031.stdout:1058 read 0x34a9be7 thru 0x34ad472 (0x388c bytes) 2024-05-08T08:25:22.641 INFO:teuthology.orchestra.run.smithi031.stdout:1059 read 0xe240be9 thru 0xe24cf6e (0xc386 bytes) 2024-05-08T08:25:22.641 INFO:teuthology.orchestra.run.smithi031.stdout:1060 writesame 0x3a78860 thru 0x3a7c3e9 data_size 0x1(0x3b8a bytes) 2024-05-08T08:25:22.662 INFO:teuthology.orchestra.run.smithi031.stdout:1061 trunc from 0xeb026a0 to 0x396d3f7 2024-05-08T08:25:22.675 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.675 INFO:teuthology.orchestra.run.smithi031.stdout:1063 write 0x1da88aa thru 0x1daa4ea (0x1c41 bytes) 2024-05-08T08:25:22.694 INFO:teuthology.orchestra.run.smithi031.stdout:1064 write 0xa8a4d59 thru 0xa8aad63 (0x600b bytes) 2024-05-08T08:25:22.695 INFO:teuthology.orchestra.run.smithi031.stdout:1066 read 0x9da7922 thru 0x9dadf55 (0x6634 bytes) 2024-05-08T08:25:22.695 INFO:teuthology.orchestra.run.smithi031.stdout:1067 punch from 0x32fff8b to 0x3305df7, (0x5e6c bytes) 2024-05-08T08:25:22.698 INFO:teuthology.orchestra.run.smithi031.stdout:1070 trunc from 0xa8aad64 to 0xe20a8f0 2024-05-08T08:25:22.702 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:22.702 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.703 INFO:teuthology.orchestra.run.smithi031.stdout:1071 punch from 0x2118c14 to 0x2126900, (0xdcec bytes) 2024-05-08T08:25:22.703 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.703 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:22.703 INFO:teuthology.orchestra.run.smithi031.stdout:1074 write 0x28e176 thru 0x29902a (0xaeb5 bytes) 2024-05-08T08:25:22.711 INFO:teuthology.orchestra.run.smithi031.stdout:1075 writesame 0xcfa93cf thru 0xcfad3d5 data_size 0x1(0x4007 bytes) 2024-05-08T08:25:22.713 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:22.718 INFO:teuthology.orchestra.run.smithi031.stdout:1077 read 0x9fc0f47 thru 0x9fc1e5e (0xf18 bytes) 2024-05-08T08:25:22.718 INFO:teuthology.orchestra.run.smithi031.stdout:1078 writesame 0xcdfdbe3 thru 0xce0982c data_size 0x4d(0xbc4a bytes) 2024-05-08T08:25:22.722 INFO:teuthology.orchestra.run.smithi031.stdout:1079 write 0xdcd98fc thru 0xdcdf542 (0x5c47 bytes) 2024-05-08T08:25:22.725 INFO:teuthology.orchestra.run.smithi031.stdout:1080 read 0x10029b4 thru 0x1007356 (0x49a3 bytes) 2024-05-08T08:25:22.725 INFO:teuthology.orchestra.run.smithi031.stdout:1081 read 0x5d34192 thru 0x5d42989 (0xe7f8 bytes) 2024-05-08T08:25:22.726 INFO:teuthology.orchestra.run.smithi031.stdout:1082 clone 9 order 25 su 2097152 sc 9 2024-05-08T08:25:22.753 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:22.760+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:22.829 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:22.836+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:22.859 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:22.859 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.867 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:22.930 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:22.938+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:22.973 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:22.973 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:22.981 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:23.030 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:23.037+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:23.091 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:23.091 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:23.102 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:23.124 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone8 from 0xe20a8f0 (overlap 0x1ef4959) to 0x0 2024-05-08T08:25:23.131 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:23.138+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:23.231 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:23.239+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:23.266 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:23.266 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:23.276 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:23.332 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:23.339+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:23.351 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-05-08T08:25:23.394 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:23.394 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:23.404 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:23.433 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:23.441+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:23.469 INFO:teuthology.orchestra.run.smithi031.stdout:1083 write 0xa54856c thru 0xa552aaa (0xa53f bytes) 2024-05-08T08:25:23.501 INFO:teuthology.orchestra.run.smithi031.stdout:1087 trunc from 0xe20a8f0 to 0xaddd29c 2024-05-08T08:25:23.519 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:23.519 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:23.532 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:23.534 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:23.540+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:23.557 INFO:teuthology.orchestra.run.smithi031.stdout:1089 read 0xac67610 thru 0xac71d40 (0xa731 bytes) 2024-05-08T08:25:23.565 INFO:teuthology.orchestra.run.smithi031.stdout:1090 write 0xc1bd705 thru 0xc1c80e3 (0xa9df bytes) 2024-05-08T08:25:23.567 INFO:teuthology.orchestra.run.smithi031.stdout:1091 punch from 0x27c7369 to 0x27d72ff, (0xff96 bytes) 2024-05-08T08:25:23.567 INFO:teuthology.orchestra.run.smithi031.stdout:1093 write 0xb01b970 thru 0xb0247b6 (0x8e47 bytes) 2024-05-08T08:25:23.568 INFO:teuthology.orchestra.run.smithi031.stdout:1094 write 0x23ae14e thru 0x23b4746 (0x65f9 bytes) 2024-05-08T08:25:23.570 INFO:teuthology.orchestra.run.smithi031.stdout:1096 punch from 0x1bd745e to 0x1bddf0c, (0x6aae bytes) 2024-05-08T08:25:23.571 INFO:teuthology.orchestra.run.smithi031.stdout:1097 write 0x418ce2c thru 0x4196e77 (0xa04c bytes) 2024-05-08T08:25:23.572 INFO:teuthology.orchestra.run.smithi031.stdout: marking collection for removal 2024-05-08T08:25:23.572 INFO:teuthology.orchestra.run.smithi031.stdout:setting '_remove' omap key 2024-05-08T08:25:23.573 INFO:teuthology.orchestra.run.smithi031.stdout:1098 read 0xa4794b8 thru 0xa47d18f (0x3cd8 bytes) 2024-05-08T08:25:23.573 INFO:teuthology.orchestra.run.smithi031.stdout:finish_remove_pgs 3.1a_head removing 3.1a 2024-05-08T08:25:23.578 INFO:teuthology.orchestra.run.smithi031.stdout:Remove successful 2024-05-08T08:25:23.579 INFO:teuthology.orchestra.run.smithi031.stdout:1099 write 0xd0cc65b thru 0xd0d5689 (0x902f bytes) 2024-05-08T08:25:23.580 INFO:teuthology.orchestra.run.smithi031.stdout:1100 write 0x6b8b20f thru 0x6b92246 (0x7038 bytes) 2024-05-08T08:25:23.582 INFO:teuthology.orchestra.run.smithi031.stdout:1101 writesame 0x27a909a thru 0x27b0fd6 data_size 0x1(0x7f3d bytes) 2024-05-08T08:25:23.597 INFO:teuthology.orchestra.run.smithi031.stdout:1102 write 0xb279ca5 thru 0xb2802ff (0x665b bytes) 2024-05-08T08:25:23.598 INFO:teuthology.orchestra.run.smithi031.stdout:1104 read 0xa3044be thru 0xa30e48a (0x9fcd bytes) 2024-05-08T08:25:23.599 INFO:teuthology.orchestra.run.smithi031.stdout:1105 read 0xa24bfa1 thru 0xa24c45a (0x4ba bytes) 2024-05-08T08:25:23.599 INFO:teuthology.orchestra.run.smithi031.stdout:1106 write 0x7201cf1 thru 0x720bbfc (0x9f0c bytes) 2024-05-08T08:25:23.601 INFO:teuthology.orchestra.run.smithi031.stdout:1107 read 0x1ed7a15 thru 0x1edd569 (0x5b55 bytes) 2024-05-08T08:25:23.609 INFO:teuthology.orchestra.run.smithi031.stdout:1108 write 0xdb9a4fb thru 0xdb9f55c (0x5062 bytes) 2024-05-08T08:25:23.611 INFO:teuthology.orchestra.run.smithi031.stdout:1110 writesame 0x5e7a3dc thru 0x5e7e4e7 data_size 0x2e(0x410c bytes) 2024-05-08T08:25:23.620 INFO:teuthology.orchestra.run.smithi031.stdout:1111 write 0xeab520d thru 0xeabd351 (0x8145 bytes) 2024-05-08T08:25:23.628 INFO:teuthology.orchestra.run.smithi031.stdout:1112 trunc from 0xeabd352 to 0x96ae5a6 2024-05-08T08:25:23.635 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:23.642+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:23.635 INFO:teuthology.orchestra.run.smithi031.stdout:1114 write 0x2cd09c8 thru 0x2cd16e8 (0xd21 bytes) 2024-05-08T08:25:23.644 INFO:teuthology.orchestra.run.smithi031.stdout:1115 write 0x9cbdd2a thru 0x9cc0f5a (0x3231 bytes) 2024-05-08T08:25:23.652 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:23.652 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:23.656 INFO:teuthology.orchestra.run.smithi031.stdout:1116 trunc from 0x9cc0f5b to 0x546a00 2024-05-08T08:25:23.663 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:23.700 INFO:teuthology.orchestra.run.smithi031.stdout:1117 writesame 0x94696f7 thru 0x9469f91 data_size 0x1(0x89b bytes) 2024-05-08T08:25:23.701 INFO:teuthology.orchestra.run.smithi031.stdout:1118 punch from 0x460475d to 0x4608729, (0x3fcc bytes) 2024-05-08T08:25:23.701 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.701 INFO:teuthology.orchestra.run.smithi031.stdout:1120 read 0x7499175 thru 0x74a118e (0x801a bytes) 2024-05-08T08:25:23.702 INFO:teuthology.orchestra.run.smithi031.stdout:1121 read 0x299267c thru 0x299bbfe (0x9583 bytes) 2024-05-08T08:25:23.715 INFO:teuthology.orchestra.run.smithi031.stdout:1122 writesame 0xe421c10 thru 0xe42f783 data_size 0x6a(0xdb74 bytes) 2024-05-08T08:25:23.715 INFO:teuthology.orchestra.run.smithi031.stdout:1125 writesame 0x96925d5 thru 0x969ad77 data_size 0x1(0x87a3 bytes) 2024-05-08T08:25:23.733 INFO:teuthology.orchestra.run.smithi031.stdout:1126 read 0x6cc052f thru 0x6cc75ca (0x709c bytes) 2024-05-08T08:25:23.736 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:23.743+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:23.737 INFO:teuthology.orchestra.run.smithi031.stdout:1127 trunc from 0xe42f784 to 0xb411d40 2024-05-08T08:25:23.751 INFO:teuthology.orchestra.run.smithi031.stdout:1128 writesame 0x997f534 thru 0x998743b data_size 0x3c(0x7f08 bytes) 2024-05-08T08:25:23.754 INFO:teuthology.orchestra.run.smithi031.stdout:1129 write 0x9b8823c thru 0x9b9706f (0xee34 bytes) 2024-05-08T08:25:23.758 INFO:teuthology.orchestra.run.smithi031.stdout:1130 read 0xa619c0d thru 0xa61a0f7 (0x4eb bytes) 2024-05-08T08:25:23.758 INFO:teuthology.orchestra.run.smithi031.stdout:1132 read 0xac27ace thru 0xac3361c (0xbb4f bytes) 2024-05-08T08:25:23.759 INFO:teuthology.orchestra.run.smithi031.stdout:1133 read 0x2e79e2a thru 0x2e85f51 (0xc128 bytes) 2024-05-08T08:25:23.759 INFO:teuthology.orchestra.run.smithi031.stdout:1134 punch from 0x480bbdf to 0x481362a, (0x7a4b bytes) 2024-05-08T08:25:23.773 INFO:teuthology.orchestra.run.smithi031.stdout:1135 writesame 0xe96ec99 thru 0xe973aaa data_size 0x3(0x4e12 bytes) 2024-05-08T08:25:23.773 INFO:teuthology.orchestra.run.smithi031.stdout:1140 writesame 0x3875478 thru 0x3879095 data_size 0xa(0x3c1e bytes) 2024-05-08T08:25:23.774 INFO:teuthology.orchestra.run.smithi031.stdout:1141 read 0xe76a492 thru 0xe77961f (0xf18e bytes) 2024-05-08T08:25:23.775 INFO:teuthology.orchestra.run.smithi031.stdout:1142 writesame 0xaf9292a thru 0xaf93632 data_size 0x1(0xd09 bytes) 2024-05-08T08:25:23.779 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.779 INFO:teuthology.orchestra.run.smithi031.stdout:1145 punch from 0x46f043 to 0x47899c, (0x9959 bytes) 2024-05-08T08:25:23.779 INFO:teuthology.orchestra.run.smithi031.stdout:1148 writesame 0x688c16a thru 0x688f8d7 data_size 0x2b(0x376e bytes) 2024-05-08T08:25:23.780 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:23.780 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:23.784 INFO:teuthology.orchestra.run.smithi031.stdout:1149 write 0x1aeb4e thru 0x1bea19 (0xfecc bytes) 2024-05-08T08:25:23.786 INFO:teuthology.orchestra.run.smithi031.stdout:1151 read 0xd1cd92a thru 0xd1d0754 (0x2e2b bytes) 2024-05-08T08:25:23.787 INFO:teuthology.orchestra.run.smithi031.stdout:1152 write 0x3d7ccc0 thru 0x3d816a7 (0x49e8 bytes) 2024-05-08T08:25:23.790 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:23.791 INFO:teuthology.orchestra.run.smithi031.stdout:1153 writesame 0x4d6325c thru 0x4d6ac1f data_size 0x2(0x79c4 bytes) 2024-05-08T08:25:23.805 INFO:teuthology.orchestra.run.smithi031.stdout:1154 trunc from 0xe973aab to 0x7e9b344 2024-05-08T08:25:23.822 INFO:teuthology.orchestra.run.smithi031.stdout:1155 trunc from 0x7e9b344 to 0xea8e0d9 2024-05-08T08:25:23.833 INFO:teuthology.orchestra.run.smithi031.stdout:1156 read 0x588ba5c thru 0x5899ff7 (0xe59c bytes) 2024-05-08T08:25:23.833 INFO:teuthology.orchestra.run.smithi031.stdout:1157 read 0xc7d727b thru 0xc7e34aa (0xc230 bytes) 2024-05-08T08:25:23.833 INFO:teuthology.orchestra.run.smithi031.stdout:1159 writesame 0x2bd41ca thru 0x2be2f28 data_size 0x1(0xed5f bytes) 2024-05-08T08:25:23.837 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:23.845+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:23.862 INFO:teuthology.orchestra.run.smithi031.stdout:1160 read 0x46665f8 thru 0x466842b (0x1e34 bytes) 2024-05-08T08:25:23.862 INFO:teuthology.orchestra.run.smithi031.stdout:1161 write 0xb6810ab thru 0xb686857 (0x57ad bytes) 2024-05-08T08:25:23.864 INFO:teuthology.orchestra.run.smithi031.stdout:1162 read 0x71ce1a9 thru 0x71dba36 (0xd88e bytes) 2024-05-08T08:25:23.865 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.865 INFO:teuthology.orchestra.run.smithi031.stdout:1164 writesame 0x44f88d7 thru 0x4507412 data_size 0xa(0xeb3c bytes) 2024-05-08T08:25:23.867 INFO:teuthology.orchestra.run.smithi031.stdout:1165 write 0x1c7fb6b thru 0x1c8f886 (0xfd1c bytes) 2024-05-08T08:25:23.869 INFO:teuthology.orchestra.run.smithi031.stdout:1167 punch from 0x7292bd8 to 0x7298990, (0x5db8 bytes) 2024-05-08T08:25:23.870 INFO:teuthology.orchestra.run.smithi031.stdout:1168 write 0xe64ce53 thru 0xe6517e7 (0x4995 bytes) 2024-05-08T08:25:23.871 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.871 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.871 INFO:teuthology.orchestra.run.smithi031.stdout:1171 write 0xe600264 thru 0xe609602 (0x939f bytes) 2024-05-08T08:25:23.872 INFO:teuthology.orchestra.run.smithi031.stdout:1172 write 0xdbad639 thru 0xdbbd50b (0xfed3 bytes) 2024-05-08T08:25:23.874 INFO:teuthology.orchestra.run.smithi031.stdout:1173 write 0x8e7f741 thru 0x8e83803 (0x40c3 bytes) 2024-05-08T08:25:23.882 INFO:teuthology.orchestra.run.smithi031.stdout:1176 trunc from 0xea8e0d9 to 0xb14565e 2024-05-08T08:25:23.893 INFO:teuthology.orchestra.run.smithi031.stdout:1177 flatten 2024-05-08T08:25:23.901 INFO:teuthology.orchestra.run.smithi031.stdout:1178 writesame 0x8662b6 thru 0x86bd1e data_size 0x5(0x5a69 bytes) 2024-05-08T08:25:23.907 INFO:teuthology.orchestra.run.smithi031.stdout:1179 read 0x7f1ee89 thru 0x7f2a744 (0xb8bc bytes) 2024-05-08T08:25:23.907 INFO:teuthology.orchestra.run.smithi031.stdout:1180 write 0xae9d402 thru 0xaea3a83 (0x6682 bytes) 2024-05-08T08:25:23.907 INFO:teuthology.orchestra.run.smithi031.stdout:1181 punch from 0x2aed5a3 to 0x2af816a, (0xabc7 bytes) 2024-05-08T08:25:23.907 INFO:teuthology.orchestra.run.smithi031.stdout:1182 write 0x8a72f93 thru 0x8a808a1 (0xd90f bytes) 2024-05-08T08:25:23.907 INFO:teuthology.orchestra.run.smithi031.stdout:1184 read 0x30ef513 thru 0x30f1f7c (0x2a6a bytes) 2024-05-08T08:25:23.907 INFO:teuthology.orchestra.run.smithi031.stdout:1185 punch from 0x9595993 to 0x959f96e, (0x9fdb bytes) 2024-05-08T08:25:23.908 INFO:teuthology.orchestra.run.smithi031.stdout:1186 punch from 0x2990385 to 0x2996156, (0x5dd1 bytes) 2024-05-08T08:25:23.915 INFO:teuthology.orchestra.run.smithi031.stdout:1187 trunc from 0xb14565e to 0x366d24a 2024-05-08T08:25:23.925 INFO:teuthology.orchestra.run.smithi031.stdout:1189 punch from 0x15876ca to 0x158f127, (0x7a5d bytes) 2024-05-08T08:25:23.925 INFO:teuthology.orchestra.run.smithi031.stdout:1191 read 0xd70849 thru 0xd76c1e (0x63d6 bytes) 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:1194 punch from 0x29fceaf to 0x2a03f50, (0x70a1 bytes) 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:1195 punch from 0x69fa7b to 0x6a1c08, (0x218d bytes) 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:1196 read 0xf2cf8 thru 0xfdd65 (0xb06e bytes) 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.926 INFO:teuthology.orchestra.run.smithi031.stdout:1201 read 0xa32911 thru 0xa3ff68 (0xd658 bytes) 2024-05-08T08:25:23.927 INFO:teuthology.orchestra.run.smithi031.stdout:1202 punch from 0x1c922b4 to 0x1c97aec, (0x5838 bytes) 2024-05-08T08:25:23.928 INFO:teuthology.orchestra.run.smithi031.stdout:1203 trunc from 0x366d24a to 0x53cf830 2024-05-08T08:25:23.936 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.936 INFO:teuthology.orchestra.run.smithi031.stdout:1205 write 0x6110d4 thru 0x61bc94 (0xabc1 bytes) 2024-05-08T08:25:23.939 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:23.946+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:23.945 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:23.945 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:23.946 INFO:teuthology.orchestra.run.smithi031.stdout:1206 writesame 0x77ed9ab thru 0x77faef2 data_size 0x9c(0xd548 bytes) 2024-05-08T08:25:23.948 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:23.948 INFO:teuthology.orchestra.run.smithi031.stdout:1208 write 0x58fea3c thru 0x590da8f (0xf054 bytes) 2024-05-08T08:25:23.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:23.958 INFO:teuthology.orchestra.run.smithi031.stdout:1209 writesame 0x9bf5001 thru 0x9bfa0e5 data_size 0x3(0x50e5 bytes) 2024-05-08T08:25:23.960 INFO:teuthology.orchestra.run.smithi031.stdout:1210 punch from 0x818b8eb to 0x8194d57, (0x946c bytes) 2024-05-08T08:25:23.960 INFO:teuthology.orchestra.run.smithi031.stdout:1214 writesame 0x658b52c thru 0x658d595 data_size 0x9(0x206a bytes) 2024-05-08T08:25:23.962 INFO:teuthology.orchestra.run.smithi031.stdout:1216 read 0x20ea08d thru 0x20efd81 (0x5cf5 bytes) 2024-05-08T08:25:23.962 INFO:teuthology.orchestra.run.smithi031.stdout:1217 write 0x2c9f0d7 thru 0x2cad348 (0xe272 bytes) 2024-05-08T08:25:23.973 INFO:teuthology.orchestra.run.smithi031.stdout:1218 trunc from 0x9bfa0e6 to 0x1deb17f 2024-05-08T08:25:23.980 INFO:teuthology.orchestra.run.smithi031.stdout:1220 punch from 0x1a4d0b6 to 0x1a56835, (0x977f bytes) 2024-05-08T08:25:23.980 INFO:teuthology.orchestra.run.smithi031.stdout:1221 read 0x98bf76 thru 0x98c72e (0x7b9 bytes) 2024-05-08T08:25:24.000 INFO:teuthology.orchestra.run.smithi031.stdout:1222 write 0xd1fb6bf thru 0xd1fe3b8 (0x2cfa bytes) 2024-05-08T08:25:24.002 INFO:teuthology.orchestra.run.smithi031.stdout:1223 punch from 0x8552a95 to 0x85586d6, (0x5c41 bytes) 2024-05-08T08:25:24.003 INFO:teuthology.orchestra.run.smithi031.stdout:1224 read 0x3f52c54 thru 0x3f61469 (0xe816 bytes) 2024-05-08T08:25:24.003 INFO:teuthology.orchestra.run.smithi031.stdout:1226 clone 10 order 20 su 65536 sc 11 2024-05-08T08:25:24.011 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:25:24.039 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:24.046+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:24.078 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:24.078 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:24.078 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:24.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:24.102 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1a.1 2024-05-08T08:25:24.144 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:24.151+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:24.171 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:24.190 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:24.207 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:24.207 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:24.208 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:24.208 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:24.210 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:24.210 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:24.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:24.229 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:24.229 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:24.242 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:24.248+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:24.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:24.251 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:24.267 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-05-08T08:25:24.268 INFO:teuthology.orchestra.run.smithi031.stdout:1227 write 0x3d23f63 thru 0x3d2ad19 (0x6db7 bytes) 2024-05-08T08:25:24.272 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:24.273 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:24.277 INFO:teuthology.orchestra.run.smithi031.stdout:1229 writesame 0x8715fb9 thru 0x871f01a data_size 0x1(0x9062 bytes) 2024-05-08T08:25:24.292 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:24.293 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:24.296 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:24.297 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:24.297 INFO:teuthology.orchestra.run.smithi031.stdout:1230 read 0x25fb614 thru 0x25ffc67 (0x4654 bytes) 2024-05-08T08:25:24.297 INFO:teuthology.orchestra.run.smithi031.stdout:1231 writesame 0x7e051ec thru 0x7e11434 data_size 0x3b(0xc249 bytes) 2024-05-08T08:25:24.306 INFO:teuthology.orchestra.run.smithi031.stdout:1232 writesame 0x568304e thru 0x568369c data_size 0x13(0x64f bytes) 2024-05-08T08:25:24.310 INFO:teuthology.orchestra.run.smithi031.stdout:1234 read 0x7b8f0a thru 0x7c83ca (0xf4c1 bytes) 2024-05-08T08:25:24.311 INFO:teuthology.orchestra.run.smithi031.stdout:1235 write 0x447cf15 thru 0x448718d (0xa279 bytes) 2024-05-08T08:25:24.317 INFO:teuthology.orchestra.run.smithi031.stdout:1236 write 0xf83062 thru 0xf92329 (0xf2c8 bytes) 2024-05-08T08:25:24.326 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.337 INFO:teuthology.orchestra.run.smithi031.stdout:1239 trunc from 0xd1fe3b9 to 0x15c11b2 2024-05-08T08:25:24.342 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:24.349+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:24.363 INFO:teuthology.orchestra.run.smithi031.stdout:1240 writesame 0x29db7be thru 0x29dbd0f data_size 0x3(0x552 bytes) 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6562 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4386873189999996, 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024048491000000002, 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6562 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931270+0000", 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.19e-06 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931276+0000", 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4790000000000004e-06 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931286+0000", 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.044e-05 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931292+0000", 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931346+0000", 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4274999999999998e-05 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931398+0000", 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1999999999999997e-05 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.935504+0000", 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041055170000000004 2024-05-08T08:25:24.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955221+0000", 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019717443000000001 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955234+0000", 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2191999999999999e-05 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955271+0000", 2024-05-08T08:25:24.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7008000000000002e-05 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955317+0000", 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6796999999999999e-05 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7004 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7077888~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:24.370 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1817374709999999, 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016625803000000001, 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7004 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:24.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:25:24.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188222+0000", 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7219999999999999e-06 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188229+0000", 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6359999999999997e-06 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188234+0000", 2024-05-08T08:25:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.502e-06 2024-05-08T08:25:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203959+0000", 2024-05-08T08:25:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015725330999999999 2024-05-08T08:25:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204038+0000", 2024-05-08T08:25:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8379000000000003e-05 2024-05-08T08:25:24.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204845+0000", 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080675500000000004 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7020 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1816482720000003, 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017204636999999998, 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7020 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:24.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188311+0000", 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188317+0000", 2024-05-08T08:25:24.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7959999999999998e-06 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188322+0000", 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204639+0000", 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016316714 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204730+0000", 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0771000000000002e-05 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205513+0000", 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078277700000000002 2024-05-08T08:25:24.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7021 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3145728~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1816272950000002, 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017850441000000002, 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.384 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7021 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:24.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6199999999999999e-07 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188331+0000", 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1069999999999999e-06 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188338+0000", 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188343+0000", 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-05-08T08:25:24.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205565+0000", 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017221841000000002 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205665+0000", 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9530000000000004e-05 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206179+0000", 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051477999999999995 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7022 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 5767168~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1816061750000006, 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018456931999999999, 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7022 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188352+0000", 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188358+0000", 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188363+0000", 2024-05-08T08:25:24.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206229+0000", 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017865941 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206327+0000", 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7888999999999997e-05 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206807+0000", 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048019000000000002 2024-05-08T08:25:24.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7038 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1815002710000009, 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019193815999999999, 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7038 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188457+0000", 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188461+0000", 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188469+0000", 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188477+0000", 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3559999999999993e-06 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206829+0000", 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018352456 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206928+0000", 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8981999999999995e-05 2024-05-08T08:25:24.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207650+0000", 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072148600000000002 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7039 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1814675300000008, 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491153000000001, 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7039 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:25:24.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188492+0000", 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188499+0000", 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7469999999999999e-06 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188505+0000", 2024-05-08T08:25:24.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207495+0000", 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990683000000001 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207577+0000", 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.174e-05 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207980+0000", 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040296899999999998 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7040 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1814442409999995, 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020064733000000001, 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7040 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188516+0000", 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999999e-06 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188522+0000", 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188527+0000", 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208019+0000", 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491871000000001 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208118+0000", 2024-05-08T08:25:24.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8855000000000001e-05 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208577+0000", 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000458784 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7056 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1813634180000001, 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020583127999999999, 2024-05-08T08:25:24.407 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188595+0000", 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0679999999999999e-06 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188603+0000", 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999997e-06 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188609+0000", 2024-05-08T08:25:24.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-05-08T08:25:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208623+0000", 2024-05-08T08:25:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0200141 2024-05-08T08:25:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208713+0000", 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9806e-05 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209176+0000", 2024-05-08T08:25:24.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046324300000000001 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7057 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4718592~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1813103009999999, 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021149303000000001, 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.413 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188647+0000", 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188650+0000", 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7100000000000001e-06 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188657+0000", 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188665+0000", 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4410000000000004e-06 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209211+0000", 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020546426999999999 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209299+0000", 2024-05-08T08:25:24.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8139000000000004e-05 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209795+0000", 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049584800000000001 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7058 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7340032~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:24.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.181279988, 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021706842, 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7058 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188677+0000", 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188681+0000", 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188688+0000", 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.494e-06 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188693+0000", 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209838+0000", 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021144677000000001 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209901+0000", 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3322000000000001e-05 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210383+0000", 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048203800000000001 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7073 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2359296~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1811844790000006, 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022130316000000001, 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188775+0000", 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188782+0000", 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8630000000000008e-06 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188787+0000", 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210421+0000", 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021633454999999999 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210481+0000", 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9911000000000003e-05 2024-05-08T08:25:24.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210902+0000", 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042136000000000002 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7074 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4980736~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1811621829999996, 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022761245999999999, 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:24.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188795+0000", 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188797+0000", 2024-05-08T08:25:24.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188803+0000", 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188807+0000", 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210951+0000", 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022143843 2024-05-08T08:25:24.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211043+0000", 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2044000000000002e-05 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211555+0000", 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051209900000000002 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7075 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7602176~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1811422559999993, 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025123269, 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7075 2024-05-08T08:25:24.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-05-08T08:25:24.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188817+0000", 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188823+0000", 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188828+0000", 2024-05-08T08:25:24.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211585+0000", 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022757553999999999 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211640+0000", 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000002e-05 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213937+0000", 2024-05-08T08:25:24.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002297182 2024-05-08T08:25:24.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.436 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7090 3.12 3:4c3c5acc:::rbd_data.119120fdbc10.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1572864~19660800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:24.436 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:24.436 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1716681320000006, 2024-05-08T08:25:24.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017096892999999998, 2024-05-08T08:25:24.436 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.437 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.437 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.437 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7090 2024-05-08T08:25:24.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.437 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198290+0000", 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.666e-06 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198297+0000", 2024-05-08T08:25:24.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:24.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198308+0000", 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0641e-05 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198314+0000", 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7850000000000003e-06 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213971+0000", 2024-05-08T08:25:24.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015656415 2024-05-08T08:25:24.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214141+0000", 2024-05-08T08:25:24.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016982999999999999 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214349+0000", 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020874199999999999 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:24.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215032+0000", 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068226599999999995 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215223+0000", 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191263 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215286+0000", 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000004e-05 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215385+0000", 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9161999999999997e-05 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10928 3.6 3:66234790:::rbd_data.11917ede465c.0000000000000000:9 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:24.445 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.2309583550000001, 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016562112, 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10928 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139012+0000", 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3908e-05 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139020+0000", 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.96e-06 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139026+0000", 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5219999999999998e-06 2024-05-08T08:25:24.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155334+0000", 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016308699999999999 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155417+0000", 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2229000000000002e-05 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155560+0000", 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014340300000000001 2024-05-08T08:25:24.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10942 3.6 3:6636ef44:::rbd_data.119177d1b31a.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.140410+0000", 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.2295466799999999, 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016364825999999999, 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.450 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10942 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140410+0000", 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140410+0000", 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140411+0000", 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.001e-06 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140415+0000", 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6439999999999999e-06 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140426+0000", 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0746e-05 2024-05-08T08:25:24.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140433+0000", 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399999999999999e-06 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155610+0000", 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015176352000000001 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155724+0000", 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114351 2024-05-08T08:25:24.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156667+0000", 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00094309500000000002 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156704+0000", 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.676e-05 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156774+0000", 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0437000000000003e-05 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10943 3.6 3:62dc9057:::rbd_data.119177d1b31a.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.2291963690000001, 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018993326000000001, 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.455 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10943 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:24.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140761+0000", 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999999e-07 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140765+0000", 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140775+0000", 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9690000000000006e-06 2024-05-08T08:25:24.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140782+0000", 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156122+0000", 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015340006999999999 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156209+0000", 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7249000000000004e-05 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159642+0000", 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034333430000000002 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159700+0000", 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8078999999999997e-05 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159753+0000", 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3019000000000002e-05 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:13015 3.1 3:828b8991:::rbd_data.1191c87c6f01.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 20971520~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.525182+0000", 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.84477428700000001, 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016529993999999999, 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13015 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.525182+0000", 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.525182+0000", 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.525184+0000", 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527730+0000", 2024-05-08T08:25:24.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025466989999999999 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527754+0000", 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3634000000000001e-05 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527766+0000", 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2228999999999999e-05 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527848+0000", 2024-05-08T08:25:24.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1841000000000005e-05 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.528021+0000", 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017315700000000001 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.528461+0000", 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00043973200000000002 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:24.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535762+0000", 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073014610000000004 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.541578+0000", 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005815705 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.541655+0000", 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7076000000000005e-05 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.541712+0000", 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6826000000000001e-05 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:13022 3.17 3:ee16c3de:::rbd_data.1191c87c6f01.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 20828828~4380244] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.83602191199999998, 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020689712999999998, 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13022 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533938+0000", 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9879999999999998e-06 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534933+0000", 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099514400000000011 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534955+0000", 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1225000000000001e-05 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534965+0000", 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597999999999999e-05 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535030+0000", 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5068999999999997e-05 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535157+0000", 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126501 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535401+0000", 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024435700000000001 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.540584+0000", 2024-05-08T08:25:24.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051824109999999996 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554524+0000", 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013939890999999999 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554584+0000", 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554624+0000", 2024-05-08T08:25:24.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0028999999999999e-05 2024-05-08T08:25:24.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.472 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.472 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.472 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:24.473 INFO:teuthology.orchestra.run.smithi031.stdout:1241 write 0x5668d0e thru 0x566fbfa (0x6eed bytes) 2024-05-08T08:25:24.474 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1243 writesame 0x6fa55e3 thru 0x6fb55ad data_size 0x1(0xffcb bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1244 writesame 0x4312494 thru 0x431e605 data_size 0xb(0xc172 bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1245 trunc from 0x6fb55ae to 0x25fa8c1 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1246 read 0x2ce36b thru 0x2d6322 (0x7fb8 bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1247 writesame 0x323be04 thru 0x324a671 data_size 0x8d(0xe86e bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1248 read 0x1bd7a44 thru 0x1be4f1f (0xd4dc bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1252 trunc from 0x324a672 to 0x4f2569c 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1253 punch from 0x213d223 to 0x213eb28, (0x1905 bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1254 read 0xaa7cfd thru 0xaabc51 (0x3f55 bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1255 read 0x354b65e thru 0x354f130 (0x3ad3 bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1256 read 0x64d56b thru 0x651a81 (0x4517 bytes) 2024-05-08T08:25:24.477 INFO:teuthology.orchestra.run.smithi031.stdout:1262 punch from 0x3d37b16 to 0x3d41260, (0x974a bytes) 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1263 trunc from 0x4f2569c to 0xb208051 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1267 trunc from 0xb208051 to 0x2301339 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1268 writesame 0xec1a9 thru 0xf4818 data_size 0x12(0x8670 bytes) 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1269 write 0x43370c thru 0x44175c (0xe051 bytes) 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1271 punch from 0x19783b0 to 0x19882c7, (0xff17 bytes) 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1273 punch from 0xbd3851 to 0xbd56ba, (0x1e69 bytes) 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1274 read 0x13b4218 thru 0x13b516e (0xf57 bytes) 2024-05-08T08:25:24.478 INFO:teuthology.orchestra.run.smithi031.stdout:1275 read 0xbf2644 thru 0xbf568f (0x304c bytes) 2024-05-08T08:25:24.479 INFO:teuthology.orchestra.run.smithi031.stdout:1277 punch from 0x1ef26ca to 0x1efb8a0, (0x91d6 bytes) 2024-05-08T08:25:24.479 INFO:teuthology.orchestra.run.smithi031.stdout:1279 read 0xaeb1d5 thru 0xaf9e8a (0xecb6 bytes) 2024-05-08T08:25:24.479 INFO:teuthology.orchestra.run.smithi031.stdout:1280 read 0x1759a9f thru 0x1768e32 (0xf394 bytes) 2024-05-08T08:25:24.479 INFO:teuthology.orchestra.run.smithi031.stdout:1281 read 0x1bf76bc thru 0x1bf9126 (0x1a6b bytes) 2024-05-08T08:25:24.479 INFO:teuthology.orchestra.run.smithi031.stdout:1282 write 0x3e87d9 thru 0x3f06fd (0x7f25 bytes) 2024-05-08T08:25:24.479 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:24.480+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:24.488 INFO:teuthology.orchestra.run.smithi031.stdout:1283 write 0x5b6d092 thru 0x5b6e84e (0x17bd bytes) 2024-05-08T08:25:24.492 INFO:teuthology.orchestra.run.smithi031.stdout:1284 read 0xa642da thru 0xa66efe (0x2c25 bytes) 2024-05-08T08:25:24.493 INFO:teuthology.orchestra.run.smithi031.stdout:1285 read 0x70139d thru 0x704d4f (0x39b3 bytes) 2024-05-08T08:25:24.500 INFO:teuthology.orchestra.run.smithi031.stdout:1286 write 0x6244d74 thru 0x624bbf6 (0x6e83 bytes) 2024-05-08T08:25:24.501 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.505 INFO:teuthology.orchestra.run.smithi031.stdout:1288 trunc from 0x624bbf7 to 0x32e3163 2024-05-08T08:25:24.523 INFO:teuthology.orchestra.run.smithi031.stdout:1289 write 0x55db4eb thru 0x55dece8 (0x37fe bytes) 2024-05-08T08:25:24.525 INFO:teuthology.orchestra.run.smithi031.stdout:1293 punch from 0x29faf7a to 0x2a0999d, (0xea23 bytes) 2024-05-08T08:25:24.531 INFO:teuthology.orchestra.run.smithi031.stdout:1294 trunc from 0x55dece9 to 0x9ae2047 2024-05-08T08:25:24.536 INFO:teuthology.orchestra.run.smithi031.stdout:1295 read 0x63d3c8e thru 0x63dfc44 (0xbfb7 bytes) 2024-05-08T08:25:24.537 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.545 INFO:teuthology.orchestra.run.smithi031.stdout:1297 writesame 0xbe59d7b thru 0xbe59dae data_size 0x1(0x34 bytes) 2024-05-08T08:25:24.545 INFO:teuthology.orchestra.run.smithi031.stdout:1300 read 0x6596cb4 thru 0x659b01a (0x4367 bytes) 2024-05-08T08:25:24.545 INFO:teuthology.orchestra.run.smithi031.stdout:1302 read 0xa40a84e thru 0xa40d591 (0x2d44 bytes) 2024-05-08T08:25:24.545 INFO:teuthology.orchestra.run.smithi031.stdout:1303 write 0x7e35cca thru 0x7e4145a (0xb791 bytes) 2024-05-08T08:25:24.552 INFO:teuthology.orchestra.run.smithi031.stdout:1304 trunc from 0xbe59daf to 0xdfd8aa5 2024-05-08T08:25:24.558 INFO:teuthology.orchestra.run.smithi031.stdout:1305 write 0x5856b63 thru 0x5865ca7 (0xf145 bytes) 2024-05-08T08:25:24.562 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.563 INFO:teuthology.orchestra.run.smithi031.stdout:1307 read 0xbaa9ab6 thru 0xbab5dc8 (0xc313 bytes) 2024-05-08T08:25:24.563 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.563 INFO:teuthology.orchestra.run.smithi031.stdout:1313 read 0x5138e71 thru 0x5140fa7 (0x8137 bytes) 2024-05-08T08:25:24.563 INFO:teuthology.orchestra.run.smithi031.stdout:1314 write 0x7066706 thru 0x706a178 (0x3a73 bytes) 2024-05-08T08:25:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:1315 read 0x8b74da9 thru 0x8b7fb58 (0xadb0 bytes) 2024-05-08T08:25:24.566 INFO:teuthology.orchestra.run.smithi031.stdout:1316 writesame 0xde0f5da thru 0xde155ef data_size 0x31(0x6016 bytes) 2024-05-08T08:25:24.571 INFO:teuthology.orchestra.run.smithi031.stdout:1317 writesame 0xc246e5 thru 0xc2eab8 data_size 0x5a(0xa3d4 bytes) 2024-05-08T08:25:24.573 INFO:teuthology.orchestra.run.smithi031.stdout:1318 read 0xb72de8f thru 0xb732624 (0x4796 bytes) 2024-05-08T08:25:24.573 INFO:teuthology.orchestra.run.smithi031.stdout:1320 punch from 0x935a6f9 to 0x935e526, (0x3e2d bytes) 2024-05-08T08:25:24.573 INFO:teuthology.orchestra.run.smithi031.stdout:1321 write 0x9057e12 thru 0x9059e69 (0x2058 bytes) 2024-05-08T08:25:24.574 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:24.582+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:24.592 INFO:teuthology.orchestra.run.smithi031.stdout:1324 trunc from 0xdfd8aa5 to 0xb0c1d2 2024-05-08T08:25:24.614 INFO:teuthology.orchestra.run.smithi031.stdout:1325 punch from 0xadf46 to 0xb1141, (0x31fb bytes) 2024-05-08T08:25:24.629 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:24.630 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:24.630 INFO:teuthology.orchestra.run.smithi031.stdout:1326 write 0x9a181e2 thru 0x9a1dc81 (0x5aa0 bytes) 2024-05-08T08:25:24.635 INFO:teuthology.orchestra.run.smithi031.stdout:1328 read 0x97e5804 thru 0x97e790b (0x2108 bytes) 2024-05-08T08:25:24.635 INFO:teuthology.orchestra.run.smithi031.stdout:1329 read 0x5cfcd18 thru 0x5d01789 (0x4a72 bytes) 2024-05-08T08:25:24.639 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:24.641 INFO:teuthology.orchestra.run.smithi031.stdout:1330 writesame 0xa8043c1 thru 0xa811b50 data_size 0x8(0xd790 bytes) 2024-05-08T08:25:24.651 INFO:teuthology.orchestra.run.smithi031.stdout:1332 writesame 0xd73f4ea thru 0xd74eafb data_size 0x3(0xf612 bytes) 2024-05-08T08:25:24.651 INFO:teuthology.orchestra.run.smithi031.stdout:1333 read 0x20bb15b thru 0x20c830d (0xd1b3 bytes) 2024-05-08T08:25:24.651 INFO:teuthology.orchestra.run.smithi031.stdout:1336 read 0xa4e1e18 thru 0xa4e606c (0x4255 bytes) 2024-05-08T08:25:24.651 INFO:teuthology.orchestra.run.smithi031.stdout:1337 punch from 0x7db2876 to 0x7dc1964, (0xf0ee bytes) 2024-05-08T08:25:24.656 INFO:teuthology.orchestra.run.smithi031.stdout:1338 trunc from 0xd74eafc to 0x888ec30 2024-05-08T08:25:24.674 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:24.682+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:24.679 INFO:teuthology.orchestra.run.smithi031.stdout:1339 writesame 0xebac067 thru 0xebb714c data_size 0x1(0xb0e6 bytes) 2024-05-08T08:25:24.680 INFO:teuthology.orchestra.run.smithi031.stdout:1340 writesame 0x942e997 thru 0x94337a4 data_size 0x61(0x4e0e bytes) 2024-05-08T08:25:24.688 INFO:teuthology.orchestra.run.smithi031.stdout:1341 writesame 0x96506d1 thru 0x965994e data_size 0x11(0x927e bytes) 2024-05-08T08:25:24.692 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.697 INFO:teuthology.orchestra.run.smithi031.stdout:1343 trunc from 0xebb714d to 0xa259d33 2024-05-08T08:25:24.714 INFO:teuthology.orchestra.run.smithi031.stdout:1344 writesame 0xdb9abec thru 0xdba454a data_size 0x47(0x995f bytes) 2024-05-08T08:25:24.714 INFO:teuthology.orchestra.run.smithi031.stdout:1345 read 0x9c3a3f9 thru 0x9c4815c (0xdd64 bytes) 2024-05-08T08:25:24.727 INFO:teuthology.orchestra.run.smithi031.stdout:1346 trunc from 0xdba454b to 0x2602f83 2024-05-08T08:25:24.741 INFO:teuthology.orchestra.run.smithi031.stdout:1348 writesame 0x20ed96f thru 0x20f7804 data_size 0x2(0x9e96 bytes) 2024-05-08T08:25:24.750 INFO:teuthology.orchestra.run.smithi031.stdout:1349 punch from 0x1aa1214 to 0x1aaa502, (0x92ee bytes) 2024-05-08T08:25:24.750 INFO:teuthology.orchestra.run.smithi031.stdout:1350 read 0xf97cbb thru 0xf9f8c1 (0x7c07 bytes) 2024-05-08T08:25:24.765 INFO:teuthology.orchestra.run.smithi031.stdout:1351 writesame 0x8af367a thru 0x8afe657 data_size 0x1(0xafde bytes) 2024-05-08T08:25:24.768 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:24.769 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:24.769 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:24.769 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:24.769 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:24.774 INFO:teuthology.orchestra.run.smithi031.stdout:1352 write 0xac2596d thru 0xac2b054 (0x56e8 bytes) 2024-05-08T08:25:24.775 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:24.781+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:24.779 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:24.782 INFO:teuthology.orchestra.run.smithi031.stdout:1353 trunc from 0xac2b055 to 0x5e31cea 2024-05-08T08:25:24.792 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.803 INFO:teuthology.orchestra.run.smithi031.stdout:1355 writesame 0xa4073f1 thru 0xa40cf74 data_size 0x2(0x5b84 bytes) 2024-05-08T08:25:24.803 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.803 INFO:teuthology.orchestra.run.smithi031.stdout:1357 read 0x8a0511f thru 0x8a0eb10 (0x99f2 bytes) 2024-05-08T08:25:24.803 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.803 INFO:teuthology.orchestra.run.smithi031.stdout:1359 writesame 0x7ad3dfe thru 0x7ad6702 data_size 0x1(0x2905 bytes) 2024-05-08T08:25:24.812 INFO:teuthology.orchestra.run.smithi031.stdout:1360 trunc from 0xa40cf75 to 0x8721812 2024-05-08T08:25:24.827 INFO:teuthology.orchestra.run.smithi031.stdout:1361 writesame 0x99970fe thru 0x99a18a4 data_size 0x1(0xa7a7 bytes) 2024-05-08T08:25:24.827 INFO:teuthology.orchestra.run.smithi031.stdout:1362 punch from 0x67e8473 to 0x67f0e40, (0x89cd bytes) 2024-05-08T08:25:24.833 INFO:teuthology.orchestra.run.smithi031.stdout:1363 trunc from 0x99a18a5 to 0x44428e3 2024-05-08T08:25:24.844 INFO:teuthology.orchestra.run.smithi031.stdout:1364 read 0xd6b25a thru 0xd73758 (0x84ff bytes) 2024-05-08T08:25:24.848 INFO:teuthology.orchestra.run.smithi031.stdout:1365 trunc from 0x44428e3 to 0x7bead33 2024-05-08T08:25:24.853 INFO:teuthology.orchestra.run.smithi031.stdout:1367 write 0x16d0b88 thru 0x16debae (0xe027 bytes) 2024-05-08T08:25:24.859 INFO:teuthology.orchestra.run.smithi031.stdout:1368 trunc from 0x7bead33 to 0x9f3bd62 2024-05-08T08:25:24.865 INFO:teuthology.orchestra.run.smithi031.stdout:1370 read 0x3ea457f thru 0x3ea4fc5 (0xa47 bytes) 2024-05-08T08:25:24.865 INFO:teuthology.orchestra.run.smithi031.stdout:1372 writesame 0x37c6bce thru 0x37d1c93 data_size 0x79(0xb0c6 bytes) 2024-05-08T08:25:24.869 INFO:teuthology.orchestra.run.smithi031.stdout:1373 writesame 0x1893554 thru 0x1896fae data_size 0x1(0x3a5b bytes) 2024-05-08T08:25:24.875 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:24.882+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:24.877 INFO:teuthology.orchestra.run.smithi031.stdout:1375 punch from 0x94b9552 to 0x94c2cee, (0x979c bytes) 2024-05-08T08:25:24.878 INFO:teuthology.orchestra.run.smithi031.stdout:1377 read 0x11f5da7 thru 0x11fdaa3 (0x7cfd bytes) 2024-05-08T08:25:24.878 INFO:teuthology.orchestra.run.smithi031.stdout:1378 write 0x5401cbd thru 0x5405e51 (0x4195 bytes) 2024-05-08T08:25:24.880 INFO:teuthology.orchestra.run.smithi031.stdout:1379 writesame 0x2e33873 thru 0x2e3c4cc data_size 0x5(0x8c5a bytes) 2024-05-08T08:25:24.887 INFO:teuthology.orchestra.run.smithi031.stdout:1380 read 0x1b1e1c thru 0x1bc480 (0xa665 bytes) 2024-05-08T08:25:24.888 INFO:teuthology.orchestra.run.smithi031.stdout:1381 read 0x583944e thru 0x584789d (0xe450 bytes) 2024-05-08T08:25:24.888 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.888 INFO:teuthology.orchestra.run.smithi031.stdout:1390 punch from 0x4571605 to 0x45731c6, (0x1bc1 bytes) 2024-05-08T08:25:24.888 INFO:teuthology.orchestra.run.smithi031.stdout:1392 punch from 0x40783b2 to 0x407af71, (0x2bbf bytes) 2024-05-08T08:25:24.888 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:24.894 INFO:teuthology.orchestra.run.smithi031.stdout:1394 write 0xb03efb1 thru 0xb0464d2 (0x7522 bytes) 2024-05-08T08:25:24.899 INFO:teuthology.orchestra.run.smithi031.stdout:1395 read 0x46cd551 thru 0x46d9634 (0xc0e4 bytes) 2024-05-08T08:25:24.899 INFO:teuthology.orchestra.run.smithi031.stdout:1396 punch from 0xa3450d5 to 0xa35398a, (0xe8b5 bytes) 2024-05-08T08:25:24.899 INFO:teuthology.orchestra.run.smithi031.stdout:1397 write 0x95656b1 thru 0x956cb66 (0x74b6 bytes) 2024-05-08T08:25:24.902 INFO:teuthology.orchestra.run.smithi031.stdout:1398 write 0x27d345 thru 0x285ab3 (0x876f bytes) 2024-05-08T08:25:24.908 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:24.908 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1729 3.9 e23/18 3:9695f221:::rbd_data.119116cc7dd0.00000000000002df:head v 23'70, mlcod=23'68)", 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.742163863, 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001912813, 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171945+0000", 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171947+0000", 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2610000000000001e-06 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171962+0000", 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4827999999999999e-05 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.171983+0000", 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1542000000000001e-05 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172004+0000", 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0577e-05 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172072+0000", 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8205000000000002e-05 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.172097+0000", 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5004999999999999e-05 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173793+0000", 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001696513 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.173858+0000", 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4882000000000007e-05 2024-05-08T08:25:24.913 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1805 3.4 e23/18 3:247a66e4:::rbd_data.119116cc7dd0.0000000000000088:head v 23'26, mlcod=22'24)", 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.68069064, 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002584514, 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233419+0000", 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233420+0000", 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7100000000000011e-07 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233438+0000", 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8084000000000002e-05 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233454+0000", 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5851000000000001e-05 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233463+0000", 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9819999999999997e-06 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233527+0000", 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4140999999999995e-05 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.233551+0000", 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4606000000000001e-05 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.235949+0000", 2024-05-08T08:25:24.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002397475 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.236003+0000", 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4404000000000003e-05 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1816 3.4 e23/18 3:22bef275:::rbd_data.119116cc7dd0.0000000000000147:head v 23'27, mlcod=23'25)", 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.668383091000001, 2024-05-08T08:25:24.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012053820000000001, 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245726+0000", 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245727+0000", 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0699999999999999e-06 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245745+0000", 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7932999999999999e-05 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245763+0000", 2024-05-08T08:25:24.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8139999999999999e-05 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245772+0000", 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2789999999999998e-06 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245849+0000", 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7167000000000002e-05 2024-05-08T08:25:24.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.245876+0000", 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7260999999999999e-05 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.246893+0000", 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001017015 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.246932+0000", 2024-05-08T08:25:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8516999999999997e-05 2024-05-08T08:25:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1829 3.4 e23/18 3:243982ca:::rbd_data.119116cc7dd0.00000000000001e4:head v 23'29, mlcod=23'27)", 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.657398139, 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002557368, 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256711+0000", 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256712+0000", 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1680000000000001e-06 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256722+0000", 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3910000000000004e-06 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256732+0000", 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0682999999999999e-05 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256738+0000", 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.276e-06 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256783+0000", 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5015000000000002e-05 2024-05-08T08:25:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.256796+0000", 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3745e-05 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259238+0000", 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024414409999999999 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.259269+0000", 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0648999999999997e-05 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:1903 3.4 e23/18 3:24e0ebf3:::rbd_data.119116cc7dd0.0000000000000376:head v 23'30, mlcod=23'28)", 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.588234533, 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015443570000000001, 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325875+0000", 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325876+0000", 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0020000000000001e-06 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325906+0000", 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9688000000000001e-05 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325920+0000", 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4558e-05 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325927+0000", 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5459999999999997e-06 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325980+0000", 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3029999999999999e-05 2024-05-08T08:25:24.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.325996+0000", 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5988999999999999e-05 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.327370+0000", 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001374632 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.327419+0000", 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8912000000000002e-05 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2088 3.4 e23/18 3:206618ae:::rbd_data.119116cc7dd0.0000000000000195:head v 23'37, mlcod=23'35)", 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.42702018, 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001240746, 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487089+0000", 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487090+0000", 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.189e-06 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487103+0000", 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2685e-05 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487121+0000", 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8134e-05 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487130+0000", 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.833e-06 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487193+0000", 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2898999999999995e-05 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.487210+0000", 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6946e-05 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.488287+0000", 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001077089 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.488330+0000", 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2970999999999999e-05 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2113 3.4 e23/18 3:23bb875f:::rbd_data.119116cc7dd0.00000000000000a6:head v 23'38, mlcod=23'36)", 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.403926752, 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020550019999999998, 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510183+0000", 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510184+0000", 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.387e-06 2024-05-08T08:25:24.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510206+0000", 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2084999999999999e-05 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510226+0000", 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0046000000000001e-05 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510238+0000", 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1843e-05 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510297+0000", 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8959999999999998e-05 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.510313+0000", 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5787999999999999e-05 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512193+0000", 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001879999 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.512238+0000", 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4894000000000001e-05 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2121 3.4 e23/18 3:27cd253f:::rbd_data.119116cc7dd0.0000000000000156:head v 23'39, mlcod=23'37)", 2024-05-08T08:25:24.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.3983521, 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023924910000000001, 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515757+0000", 2024-05-08T08:25:24.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515758+0000", 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6899999999999996e-07 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515777+0000", 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8817999999999998e-05 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515792+0000", 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5277999999999998e-05 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515800+0000", 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6000000000000001e-06 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515868+0000", 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7834000000000001e-05 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.515899+0000", 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1473999999999997e-05 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.518104+0000", 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0022054000000000002 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.518150+0000", 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5318000000000002e-05 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2137 3.4 e23/18 3:206618ae:::rbd_data.119116cc7dd0.0000000000000195:head v 23'41, mlcod=23'38)", 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.372604276000001, 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027337440000000002, 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:24.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541505+0000", 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541506+0000", 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8599999999999998e-07 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541511+0000", 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5550000000000002e-06 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541523+0000", 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.207e-05 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541530+0000", 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0990000000000001e-06 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541599+0000", 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8143000000000001e-05 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.541617+0000", 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8794999999999999e-05 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544188+0000", 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025705290000000002 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544239+0000", 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0967000000000002e-05 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2141 3.9 e23/18 3:9250a055:::rbd_data.119116cc7dd0.000000000000027d:head v 23'88, mlcod=23'86)", 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.369628576, 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023341719999999998, 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544481+0000", 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544482+0000", 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544487+0000", 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1590000000000001e-06 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544499+0000", 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2621e-05 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544506+0000", 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7510000000000002e-06 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544577+0000", 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0801000000000002e-05 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.544599+0000", 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1642e-05 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546771+0000", 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021727180000000001 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.546815+0000", 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3683000000000002e-05 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2207 3.9 e23/18 3:961d4b12:::rbd_data.119116cc7dd0.00000000000000cf:head v 23'91, mlcod=23'89)", 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.312672710999999, 2024-05-08T08:25:24.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001369622, 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601437+0000", 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601438+0000", 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6840000000000001e-06 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601456+0000", 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7320999999999999e-05 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601475+0000", 2024-05-08T08:25:24.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9448000000000001e-05 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601484+0000", 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3379999999999994e-06 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601554+0000", 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9820000000000006e-05 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.601581+0000", 2024-05-08T08:25:24.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6517e-05 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.602759+0000", 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001178183 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:10.602806+0000", 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7311000000000002e-05 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2329 3.0 e24/18 3:071e2661:::rbd_data.119116cc7dd0.0000000000000012:head v 24'42, mlcod=23'39)", 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.708666+0000", 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.205443824, 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0013530300000000001, 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708666+0000", 2024-05-08T08:25:24.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708666+0000", 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708667+0000", 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2419999999999999e-06 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708688+0000", 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1330000000000001e-05 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708709+0000", 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1030999999999999e-05 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708719+0000", 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8260000000000005e-06 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708783+0000", 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4226999999999993e-05 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.708805+0000", 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2065999999999998e-05 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709982+0000", 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011766179999999999 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.710019+0000", 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6690000000000003e-05 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.963 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2337 3.0 e24/18 3:04522806:::rbd_data.119116cc7dd0.000000000000001a:head v 24'44, mlcod=23'39)", 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.709397+0000", 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.204712191, 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011970329999999999, 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709397+0000", 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709397+0000", 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709398+0000", 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.37e-07 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709405+0000", 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4599999999999997e-06 2024-05-08T08:25:24.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709415+0000", 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3789999999999995e-06 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709420+0000", 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3140000000000003e-06 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709475+0000", 2024-05-08T08:25:24.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439000000000002e-05 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.709499+0000", 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3425999999999999e-05 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.710558+0000", 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001059143 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.710594+0000", 2024-05-08T08:25:24.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6134999999999999e-05 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2351 3.4 e24/18 3:23bb875f:::rbd_data.119116cc7dd0.00000000000000a6:head v 24'44, mlcod=23'41)", 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.714884+0000", 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.199225825999999, 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001342593, 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.968 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714884+0000", 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714884+0000", 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714892+0000", 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6309999999999992e-06 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714902+0000", 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9960000000000005e-06 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714922+0000", 2024-05-08T08:25:24.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0098999999999998e-05 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714933+0000", 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1204e-05 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.714993+0000", 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9670000000000003e-05 2024-05-08T08:25:24.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.715016+0000", 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3224e-05 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.716182+0000", 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001165983 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.716226+0000", 2024-05-08T08:25:24.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3785999999999997e-05 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2370 3.4 e24/18 3:237be9c1:::rbd_data.119116cc7dd0.00000000000001eb:head v 24'46, mlcod=23'42)", 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.192822906, 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029578109999999999, 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721287+0000", 2024-05-08T08:25:24.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7199999999999997e-07 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721294+0000", 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9800000000000001e-06 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721309+0000", 2024-05-08T08:25:24.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5116999999999999e-05 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9030000000000004e-06 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721383+0000", 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6111000000000005e-05 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721406+0000", 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2453e-05 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724192+0000", 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027864909999999999 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724244+0000", 2024-05-08T08:25:24.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.188609598999999, 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:25:24.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.918916061999999, 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:25:24.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:25:24.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:6441 3.0 e27/18 3:0362a1e0:::rbd_data.119152d8614e.000000000000000f:head v 27'56, mlcod=26'54)", 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.764338+0000", 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.149771649, 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001439105, 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764338+0000", 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764338+0000", 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764340+0000", 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.148e-06 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764374+0000", 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3760000000000002e-05 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764399+0000", 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4967999999999999e-05 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764411+0000", 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2541000000000001e-05 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764491+0000", 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9487000000000007e-05 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.764518+0000", 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6933000000000001e-05 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.765724+0000", 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0012066799999999999 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.765777+0000", 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2587999999999999e-05 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14046 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:24.607166+0000", 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.30694286700000001, 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001710446, 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14046 2024-05-08T08:25:24.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607166+0000", 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607166+0000", 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607168+0000", 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6589999999999999e-06 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607176+0000", 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6690000000000006e-06 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607198+0000", 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2357e-05 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607208+0000", 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5650000000000007e-06 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607273+0000", 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5481000000000005e-05 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:24.996 INFO:teuthology.orchestra.run.smithi031.stdout:1399 read 0x5f0bb88 thru 0x5f10857 (0x4cd0 bytes) 2024-05-08T08:25:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607398+0000", 2024-05-08T08:25:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000124941 2024-05-08T08:25:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.608877+0000", 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001478774 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14050 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:24.607245+0000", 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.30686427999999999, 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016374880000000001, 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:24.997 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14050 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607245+0000", 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607245+0000", 2024-05-08T08:25:24.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607246+0000", 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8299999999999997e-07 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607249+0000", 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.777e-06 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607258+0000", 2024-05-08T08:25:24.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5250000000000005e-06 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607265+0000", 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.562e-06 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607328+0000", 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3504999999999996e-05 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.607480+0000", 2024-05-08T08:25:25.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015238900000000001 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:24.608883+0000", 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0014021470000000001 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.001 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:25.002 INFO:teuthology.orchestra.run.smithi031.stdout:1400 trunc from 0xb0464d3 to 0xb609467 2024-05-08T08:25:25.002 INFO:teuthology.orchestra.run.smithi031.stdout:1401 read 0x4e1d7c7 thru 0x4e1ed6a (0x15a4 bytes) 2024-05-08T08:25:25.002 INFO:teuthology.orchestra.run.smithi031.stdout:1402 trunc from 0xb609467 to 0xaae3a91 2024-05-08T08:25:25.003 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:1403 write 0x89c2412 thru 0x89c58ab (0x349a bytes) 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:1404 read 0x3e7fb7b thru 0x3e8f609 (0xfa8f bytes) 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:1405 write 0x299a339 thru 0x29a5a24 (0xb6ec bytes) 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:1408 write 0x1c668c5 thru 0x1c723ef (0xbb2b bytes) 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:1409 write 0xc8c3435 thru 0xc8c60c4 (0x2c90 bytes) 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:1410 read 0x3a4d31c thru 0x3a5c2b8 (0xef9d bytes) 2024-05-08T08:25:25.006 INFO:teuthology.orchestra.run.smithi031.stdout:1411 write 0x686e109 thru 0x686f0cb (0xfc3 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1412 read 0x368d809 thru 0x369bf7a (0xe772 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1413 writesame 0x5f8be48 thru 0x5f98c7d data_size 0x5(0xce36 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1417 read 0x3a05fbf thru 0x3a08333 (0x2375 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1418 trunc from 0xc8c60c5 to 0xc5fc010 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1419 write 0xb1567c5 thru 0xb162e79 (0xc6b5 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1420 punch from 0x46e12e6 to 0x46ef436, (0xe150 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1421 writesame 0xbd94857 thru 0xbd9a34f data_size 0x7(0x5af9 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1422 write 0xd557a3e thru 0xd55ccfb (0x52be bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1423 read 0x3bf7b6e thru 0x3c05007 (0xd49a bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1425 write 0x4ab4c96 thru 0x4ab6f99 (0x2304 bytes) 2024-05-08T08:25:25.007 INFO:teuthology.orchestra.run.smithi031.stdout:1426 read 0x61bc5d3 thru 0x61c7b47 (0xb575 bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:1428 read 0xcb552b6 thru 0xcb58e76 (0x3bc1 bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:1430 write 0xef07a2 thru 0xef45bb (0x3e1a bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:1431 punch from 0xafb9a1d to 0xafc8b74, (0xf157 bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:1433 writesame 0x30c3828 thru 0x30cc463 data_size 0x32(0x8c3c bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:1434 punch from 0x1c8ba12 to 0x1c91416, (0x5a04 bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:1435 read 0x4e7eb56 thru 0x4e8d77b (0xec26 bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:1436 write 0x4fa099 thru 0x5071dd (0xd145 bytes) 2024-05-08T08:25:25.008 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.009 INFO:teuthology.orchestra.run.smithi031.stdout:1439 write 0xd6b4f58 thru 0xd6bb3d4 (0x647d bytes) 2024-05-08T08:25:25.009 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:25.016+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:25.011 INFO:teuthology.orchestra.run.smithi031.stdout:1440 read 0x9de21a7 thru 0x9dedd81 (0xbbdb bytes) 2024-05-08T08:25:25.011 INFO:teuthology.orchestra.run.smithi031.stdout:1441 write 0x12532d3 thru 0x1261e26 (0xeb54 bytes) 2024-05-08T08:25:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:1442 punch from 0x1b4179 to 0x1bc0fa, (0x7f81 bytes) 2024-05-08T08:25:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:1443 read 0xa84bfe thru 0xa855fc (0x9ff bytes) 2024-05-08T08:25:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:1446 punch from 0x3be729e to 0x3bf3435, (0xc197 bytes) 2024-05-08T08:25:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:1447 read 0xba4fee9 thru 0xba5b193 (0xb2ab bytes) 2024-05-08T08:25:25.013 INFO:teuthology.orchestra.run.smithi031.stdout:1452 write 0xb8044e thru 0xb8ff84 (0xfb37 bytes) 2024-05-08T08:25:25.015 INFO:teuthology.orchestra.run.smithi031.stdout:1453 read 0xc856dea thru 0xc861661 (0xa878 bytes) 2024-05-08T08:25:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:1456 read 0x7fcb7b5 thru 0x7fda357 (0xeba3 bytes) 2024-05-08T08:25:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.016 INFO:teuthology.orchestra.run.smithi031.stdout:1459 writesame 0x19a5b3b thru 0x19b1adb data_size 0x1(0xbfa1 bytes) 2024-05-08T08:25:25.022 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:25:25.051 INFO:teuthology.orchestra.run.smithi031.stdout:1460 trunc from 0xd6bb3d5 to 0x4212cee 2024-05-08T08:25:25.069 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.070 INFO:teuthology.orchestra.run.smithi031.stdout:1464 punch from 0x3f2459d to 0x3f24935, (0x398 bytes) 2024-05-08T08:25:25.070 INFO:teuthology.orchestra.run.smithi031.stdout:1465 writesame 0x172160b thru 0x172402c data_size 0x1(0x2a22 bytes) 2024-05-08T08:25:25.093 INFO:teuthology.orchestra.run.smithi031.stdout:1467 writesame 0xce3fd54 thru 0xce47a46 data_size 0x1(0x7cf3 bytes) 2024-05-08T08:25:25.093 INFO:teuthology.orchestra.run.smithi031.stdout:1468 read 0x6d0fcbc thru 0x6d12ea4 (0x31e9 bytes) 2024-05-08T08:25:25.093 INFO:teuthology.orchestra.run.smithi031.stdout:1469 punch from 0xbc424dc to 0xbc43e92, (0x19b6 bytes) 2024-05-08T08:25:25.093 INFO:teuthology.orchestra.run.smithi031.stdout:1470 write 0x6cc8be8 thru 0x6cd1ede (0x92f7 bytes) 2024-05-08T08:25:25.099 INFO:teuthology.orchestra.run.smithi031.stdout:1471 read 0x96613ab thru 0x966340b (0x2061 bytes) 2024-05-08T08:25:25.105 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:25.112+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:25.106 INFO:teuthology.orchestra.run.smithi031.stdout:1474 write 0xe650be7 thru 0xe65461b (0x3a35 bytes) 2024-05-08T08:25:25.109 INFO:teuthology.orchestra.run.smithi031.stdout:1476 punch from 0x8d6b60b to 0x8d6e808, (0x31fd bytes) 2024-05-08T08:25:25.109 INFO:teuthology.orchestra.run.smithi031.stdout:1477 read 0x1387f2f thru 0x138a420 (0x24f2 bytes) 2024-05-08T08:25:25.109 INFO:teuthology.orchestra.run.smithi031.stdout:1478 write 0x31c543a thru 0x31c5922 (0x4e9 bytes) 2024-05-08T08:25:25.111 INFO:teuthology.orchestra.run.smithi031.stdout:1480 read 0xb5e58dc thru 0xb5f5660 (0xfd85 bytes) 2024-05-08T08:25:25.129 INFO:teuthology.orchestra.run.smithi031.stdout:1481 trunc from 0xe65461c to 0x10866d7 2024-05-08T08:25:25.146 INFO:teuthology.orchestra.run.smithi031.stdout:1482 read 0xb75fae thru 0xb7bead (0x5f00 bytes) 2024-05-08T08:25:25.150 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:25.150 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:25.151 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:25.159 INFO:teuthology.orchestra.run.smithi031.stdout:1483 write 0x892dbf7 thru 0x892f3c1 (0x17cb bytes) 2024-05-08T08:25:25.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:25.163 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.165 INFO:teuthology.orchestra.run.smithi031.stdout:1488 trunc from 0x892f3c2 to 0x684ee68 2024-05-08T08:25:25.175 INFO:teuthology.orchestra.run.smithi031.stdout:1491 writesame 0x78cd24 thru 0x7942c6 data_size 0x13(0x75a3 bytes) 2024-05-08T08:25:25.195 INFO:teuthology.orchestra.run.smithi031.stdout:1492 writesame 0xce042ec thru 0xce10164 data_size 0x1(0xbe79 bytes) 2024-05-08T08:25:25.195 INFO:teuthology.orchestra.run.smithi031.stdout:1495 read 0x7465490 thru 0x746c436 (0x6fa7 bytes) 2024-05-08T08:25:25.200 INFO:teuthology.orchestra.run.smithi031.stdout:1496 write 0xd32a6b5 thru 0xd332fd4 (0x8920 bytes) 2024-05-08T08:25:25.206 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:25.213+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:25.207 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.207 INFO:teuthology.orchestra.run.smithi031.stdout:1498 read 0x6e78113 thru 0x6e84604 (0xc4f2 bytes) 2024-05-08T08:25:25.207 INFO:teuthology.orchestra.run.smithi031.stdout:1499 punch from 0x3f02083 to 0x3f0a049, (0x7fc6 bytes) 2024-05-08T08:25:25.207 INFO:teuthology.orchestra.run.smithi031.stdout:1500 read 0x7d2ab24 thru 0x7d393e8 (0xe8c5 bytes) 2024-05-08T08:25:25.207 INFO:teuthology.orchestra.run.smithi031.stdout:1501 read 0xa48c1bf thru 0xa48de2f (0x1c71 bytes) 2024-05-08T08:25:25.207 INFO:teuthology.orchestra.run.smithi031.stdout:1502 write 0x958f2a9 thru 0x95904d4 (0x122c bytes) 2024-05-08T08:25:25.211 INFO:teuthology.orchestra.run.smithi031.stdout:1503 read 0x28f96a3 thru 0x28fea8c (0x53ea bytes) 2024-05-08T08:25:25.211 INFO:teuthology.orchestra.run.smithi031.stdout:1504 write 0x71e06d3 thru 0x71ed26c (0xcb9a bytes) 2024-05-08T08:25:25.214 INFO:teuthology.orchestra.run.smithi031.stdout:1505 write 0xb1d72a0 thru 0xb1e25c4 (0xb325 bytes) 2024-05-08T08:25:25.229 INFO:teuthology.orchestra.run.smithi031.stdout:1507 trunc from 0xd332fd5 to 0x3b83013 2024-05-08T08:25:25.240 INFO:teuthology.orchestra.run.smithi031.stdout:1508 read 0x34b7cd3 thru 0x34bd192 (0x54c0 bytes) 2024-05-08T08:25:25.241 INFO:teuthology.orchestra.run.smithi031.stdout:1509 trunc from 0x3b83013 to 0x4d53302 2024-05-08T08:25:25.255 INFO:teuthology.orchestra.run.smithi031.stdout:get_pg_num_history pg_num_history pg_num_history(e30 pg_nums {1={14=8},2={15=1},3={18=32}} deleted_pools ) 2024-05-08T08:25:25.255 INFO:teuthology.orchestra.run.smithi031.stdout:Importing pgid 3.1a 2024-05-08T08:25:25.258 INFO:teuthology.orchestra.run.smithi031.stdout:1511 write 0x9175ad8 thru 0x9177c6c (0x2195 bytes) 2024-05-08T08:25:25.260 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:581f9ee9:::rbd_data.1191f7f15880.000000000000000d:head# 2024-05-08T08:25:25.260 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 6=[]:{} 2024-05-08T08:25:25.261 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:588df1ca:::rbd_data.1191f7f15880.0000000000000030:head# 2024-05-08T08:25:25.261 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 6=[]:{} 2024-05-08T08:25:25.261 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:59a2de6e:::rbd_data.1191f7f15880.0000000000000053:head# 2024-05-08T08:25:25.261 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 6=[]:{} 2024-05-08T08:25:25.262 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5aec0c35:::rbd_object_map.119120fdbc10:head# 2024-05-08T08:25:25.262 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 0=[]:{} 2024-05-08T08:25:25.262 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5b7a5ee6:::rbd_data.1191f7f15880.000000000000003e:head# 2024-05-08T08:25:25.262 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 6=[]:{} 2024-05-08T08:25:25.262 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head# 2024-05-08T08:25:25.262 INFO:teuthology.orchestra.run.smithi031.stdout:1512 write 0x641c165 thru 0x642bf58 (0xfdf4 bytes) 2024-05-08T08:25:25.272 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 0=[]:{} 2024-05-08T08:25:25.278 INFO:teuthology.orchestra.run.smithi031.stdout:1513 writesame 0xd964d1f thru 0xd970030 data_size 0x1(0xb312 bytes) 2024-05-08T08:25:25.284 INFO:teuthology.orchestra.run.smithi031.stdout:1514 write 0xea354a7 thru 0xea388ff (0x3459 bytes) 2024-05-08T08:25:25.285 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5cae588f:::rbd_data.1191f7f15880.000000000000000b:head# 2024-05-08T08:25:25.285 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 6=[]:{} 2024-05-08T08:25:25.286 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:4# 2024-05-08T08:25:25.287 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head# 2024-05-08T08:25:25.287 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 4=[]:{4=[4]} 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:1515 read 0x17c3c73 thru 0x17cf079 (0xb407 bytes) 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:1517 punch from 0xa40a82a to 0xa40b331, (0xb07 bytes) 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:1519 read 0x20fa93c thru 0x2100b8f (0x6254 bytes) 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:1520 punch from 0x69d2900 to 0x69ddf73, (0xb673 bytes) 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:1521 read 0x4c48c1f thru 0x4c4f81b (0x6bfd bytes) 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.288 INFO:teuthology.orchestra.run.smithi031.stdout:1524 read 0xb6021c8 thru 0xb602596 (0x3cf bytes) 2024-05-08T08:25:25.289 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5cea7035:::rbd_info:head# 2024-05-08T08:25:25.289 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 0=[]:{} 2024-05-08T08:25:25.289 INFO:teuthology.orchestra.run.smithi031.stdout:1525 write 0xa21db1f thru 0xa22338f (0x5871 bytes) 2024-05-08T08:25:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5d02ff65:::rbd_data.1191f7f15880.0000000000000061:head# 2024-05-08T08:25:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 6=[]:{} 2024-05-08T08:25:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:Write #3:5d86de77:::rbd_data.1191f7f15880.0000000000000046:head# 2024-05-08T08:25:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:snapset 6=[]:{} 2024-05-08T08:25:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:write_pg epoch 29 info 3.1a( v 29'71 (0'0,29'71] local-lis/les=18/19 n=12 ec=18/18 lis/c=18/18 les/c/f=19/19/0 sis=18) 2024-05-08T08:25:25.290 INFO:teuthology.orchestra.run.smithi031.stdout:Import successful 2024-05-08T08:25:25.291 INFO:teuthology.orchestra.run.smithi031.stdout:1526 writesame 0x8bc2557 thru 0x8bc579e data_size 0x4(0x3248 bytes) 2024-05-08T08:25:25.295 INFO:teuthology.orchestra.run.smithi031.stdout:1530 punch from 0xe6c3411 to 0xe6cdcb6, (0xa8a5 bytes) 2024-05-08T08:25:25.297 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:25.297 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:25.297 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:25.297 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:25.297 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:25.307 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:25.314+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:25.310 INFO:teuthology.orchestra.run.smithi031.stdout:1531 trunc from 0xea38900 to 0x12f261e 2024-05-08T08:25:25.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:25.321 INFO:teuthology.orchestra.run.smithi031.stdout:1533 punch from 0xabf5d5 to 0xacc9a9, (0xd3d4 bytes) 2024-05-08T08:25:25.326 INFO:teuthology.orchestra.run.smithi031.stdout:1534 trunc from 0x12f261e to 0x51b9f83 2024-05-08T08:25:25.332 INFO:teuthology.orchestra.run.smithi031.stdout:1535 punch from 0x1987071 to 0x198ceb7, (0x5e46 bytes) 2024-05-08T08:25:25.339 INFO:teuthology.orchestra.run.smithi031.stdout:1537 trunc from 0x51b9f83 to 0xb4d3da9 2024-05-08T08:25:25.345 INFO:teuthology.orchestra.run.smithi031.stdout:1538 write 0x32d4935 thru 0x32dae0f (0x64db bytes) 2024-05-08T08:25:25.350 INFO:teuthology.orchestra.run.smithi031.stdout:1541 trunc from 0xb4d3da9 to 0xcc441b3 2024-05-08T08:25:25.356 INFO:teuthology.orchestra.run.smithi031.stdout:1542 read 0x6f17948 thru 0x6f24380 (0xca39 bytes) 2024-05-08T08:25:25.356 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.356 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.356 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.356 INFO:teuthology.orchestra.run.smithi031.stdout:1547 read 0x1da905 thru 0x1e1d03 (0x73ff bytes) 2024-05-08T08:25:25.356 INFO:teuthology.orchestra.run.smithi031.stdout:1549 read 0xb3e4c thru 0xb4a08 (0xbbd bytes) 2024-05-08T08:25:25.357 INFO:teuthology.orchestra.run.smithi031.stdout:1550 write 0xc6656c0 thru 0xc66824f (0x2b90 bytes) 2024-05-08T08:25:25.361 INFO:teuthology.orchestra.run.smithi031.stdout:1551 write 0x3d85286 thru 0x3d94240 (0xefbb bytes) 2024-05-08T08:25:25.367 INFO:teuthology.orchestra.run.smithi031.stdout:1553 trunc from 0xcc441b3 to 0xcee7d6c 2024-05-08T08:25:25.373 INFO:teuthology.orchestra.run.smithi031.stdout:1554 punch from 0xc4fa12f to 0xc4fa780, (0x651 bytes) 2024-05-08T08:25:25.373 INFO:teuthology.orchestra.run.smithi031.stdout:1556 punch from 0x143aa49 to 0x143f8c5, (0x4e7c bytes) 2024-05-08T08:25:25.373 INFO:teuthology.orchestra.run.smithi031.stdout:1557 read 0xf90fc0 thru 0xf96036 (0x5077 bytes) 2024-05-08T08:25:25.373 INFO:teuthology.orchestra.run.smithi031.stdout:1559 writesame 0xc79bdf9 thru 0xc7a9f12 data_size 0x1(0xe11a bytes) 2024-05-08T08:25:25.394 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.394 INFO:teuthology.orchestra.run.smithi031.stdout:1563 writesame 0xca94514 thru 0xca985e0 data_size 0x1(0x40cd bytes) 2024-05-08T08:25:25.402 INFO:teuthology.orchestra.run.smithi031.stdout:1566 punch from 0x3ec09f1 to 0x3ecf18c, (0xe79b bytes) 2024-05-08T08:25:25.402 INFO:teuthology.orchestra.run.smithi031.stdout:1568 write 0x10d658 thru 0x112eb7 (0x5860 bytes) 2024-05-08T08:25:25.408 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:25.415+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:25.408 INFO:teuthology.orchestra.run.smithi031.stdout:1570 read 0x815f06e thru 0x81693ac (0xa33f bytes) 2024-05-08T08:25:25.408 INFO:teuthology.orchestra.run.smithi031.stdout:1571 punch from 0xbd9fa3f to 0xbda756e, (0x7b2f bytes) 2024-05-08T08:25:25.408 INFO:teuthology.orchestra.run.smithi031.stdout:1572 writesame 0x7a61ad0 thru 0x7a70931 data_size 0x15(0xee62 bytes) 2024-05-08T08:25:25.419 INFO:teuthology.orchestra.run.smithi031.stdout:1573 write 0xe3a79e9 thru 0xe3af19f (0x77b7 bytes) 2024-05-08T08:25:25.424 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.424 INFO:teuthology.orchestra.run.smithi031.stdout:1575 writesame 0x86a08ee thru 0x86a5aa9 data_size 0x2(0x51bc bytes) 2024-05-08T08:25:25.429 INFO:teuthology.orchestra.run.smithi031.stdout:1576 punch from 0x323809c to 0x3247680, (0xf5e4 bytes) 2024-05-08T08:25:25.429 INFO:teuthology.orchestra.run.smithi031.stdout:1579 read 0x8cd7368 thru 0x8ce30de (0xbd77 bytes) 2024-05-08T08:25:25.429 INFO:teuthology.orchestra.run.smithi031.stdout:1580 write 0x5ea5e52 thru 0x5eaba65 (0x5c14 bytes) 2024-05-08T08:25:25.432 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.432 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xec24c1a 2024-05-08T08:25:25.432 INFO:teuthology.orchestra.run.smithi031.stdout:1583 trunc from 0xe3af1a0 to 0xec24c1a 2024-05-08T08:25:25.437 INFO:teuthology.orchestra.run.smithi031.stdout:1586 write 0x925dc69 thru 0x926a19a (0xc532 bytes) 2024-05-08T08:25:25.442 INFO:teuthology.orchestra.run.smithi031.stdout:1588 write 0x3450cf thru 0x34715a (0x208c bytes) 2024-05-08T08:25:25.443 INFO:teuthology.orchestra.run.smithi031.stdout:1589 write 0x5d385d4 thru 0x5d3c243 (0x3c70 bytes) 2024-05-08T08:25:25.445 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:25.445 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:25.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:25.445 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6425 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.74321115, 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03287702, 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6425 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708825+0000", 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7e-06 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708833+0000", 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708851+0000", 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7961e-05 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708859+0000", 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2439999999999993e-06 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708920+0000", 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1178000000000003e-05 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.709031+0000", 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011102 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.714353+0000", 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053222440000000003 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741561+0000", 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027207346 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741581+0000", 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0854000000000001e-05 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741644+0000", 2024-05-08T08:25:25.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2526000000000004e-05 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741700+0000", 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5812e-05 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6447 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.6777774, 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020412153999999998, 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6447 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:25.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774258+0000", 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774267+0000", 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8869999999999998e-06 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774285+0000", 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7399000000000001e-05 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774293+0000", 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1089999999999998e-06 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774350+0000", 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774416+0000", 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6166000000000003e-05 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.777145+0000", 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027283160000000002 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794536+0000", 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017391163000000001 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794555+0000", 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9377000000000001e-05 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794614+0000", 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8829000000000002e-05 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794669+0000", 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:25:25.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7045 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 2883584~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188874+0000", 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2631596199999997, 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019434, 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7045 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188874+0000", 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188874+0000", 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188875+0000", 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188877+0000", 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188883+0000", 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188895+0000", 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1970000000000001e-05 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207453+0000", 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018558270000000002 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207822+0000", 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036873199999999998 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208308+0000", 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048613299999999998 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7046 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 5505024~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2631316750000003, 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020079657000000001, 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7046 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188903+0000", 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188905+0000", 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188911+0000", 2024-05-08T08:25:25.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188915+0000", 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208342+0000", 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019426335999999999 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208397+0000", 2024-05-08T08:25:25.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5417000000000002e-05 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208982+0000", 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058485500000000001 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7047 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2631127979999999, 2024-05-08T08:25:25.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020213064999999999, 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7047 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2099999999999998e-07 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188923+0000", 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188929+0000", 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188934+0000", 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7360000000000001e-06 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209012+0000", 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020078822999999999 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209078+0000", 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5859999999999996e-05 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209134+0000", 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5952999999999998e-05 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7051 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2630937679999992, 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020448772, 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7051 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188941+0000", 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2800000000000003e-07 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188943+0000", 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188949+0000", 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7819999999999999e-06 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188954+0000", 2024-05-08T08:25:25.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.164e-06 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209169+0000", 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020215151000000001 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209252+0000", 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3341e-05 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209389+0000", 2024-05-08T08:25:25.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000136704 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7052 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2630746100000003, 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020652770000000001, 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.474 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7052 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188960+0000", 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188966+0000", 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5370000000000003e-06 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188972+0000", 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188977+0000", 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3939999999999998e-06 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209425+0000", 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020447775000000001 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209511+0000", 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6273999999999994e-05 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209612+0000", 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101344 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7062 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2628961489999995, 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021205383000000001, 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7062 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189140+0000", 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:25:25.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189146+0000", 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.716e-06 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189151+0000", 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5820000000000003e-06 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209642+0000", 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020491067000000002 2024-05-08T08:25:25.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209730+0000", 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8078000000000006e-05 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210343+0000", 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061346199999999995 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7063 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 4194304~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2628776049999999, 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022006687, 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7063 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189157+0000", 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1600000000000002e-07 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189159+0000", 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9570000000000001e-06 2024-05-08T08:25:25.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189164+0000", 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189169+0000", 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3680000000000004e-06 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210391+0000", 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021222382000000001 2024-05-08T08:25:25.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210478+0000", 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7168000000000003e-05 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211163+0000", 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000684693 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7064 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2628594559999993, 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022198532, 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7064 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-07 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189177+0000", 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.88e-06 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189182+0000", 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7910000000000001e-06 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189187+0000", 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-05-08T08:25:25.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211198+0000", 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022011134000000002 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211289+0000", 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0537000000000004e-05 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211373+0000", 2024-05-08T08:25:25.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4426999999999997e-05 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7068 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.262841345, 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022446079000000001, 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7068 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8599999999999998e-07 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189195+0000", 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189201+0000", 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189216+0000", 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4758e-05 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211430+0000", 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022213944999999999 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211529+0000", 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9504000000000001e-05 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211639+0000", 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010937099999999999 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7069 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.262810365, 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022643852999999999, 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.495 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7069 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189227+0000", 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6410000000000002e-06 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189234+0000", 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189244+0000", 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597e-05 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211685+0000", 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022441040999999998 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211778+0000", 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2449999999999997e-05 2024-05-08T08:25:25.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211867+0000", 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9833999999999994e-05 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7076 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2627189130000005, 2024-05-08T08:25:25.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019137119000000001, 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7076 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189317+0000", 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7489999999999999e-06 2024-05-08T08:25:25.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189322+0000", 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189327+0000", 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2719999999999999e-06 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207803+0000", 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018476037000000001 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207880+0000", 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7377000000000001e-05 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208452+0000", 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00057198700000000004 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.503 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7077 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2627003680000009, 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019800081000000001, 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7077 2024-05-08T08:25:25.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:25:25.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189336+0000", 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189342+0000", 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0220000000000001e-06 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189347+0000", 2024-05-08T08:25:25.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999998e-06 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208501+0000", 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019154612000000001 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208603+0000", 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101408 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209134+0000", 2024-05-08T08:25:25.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000530938 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7078 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2626809810000008, 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020694535, 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.508 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7078 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189355+0000", 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189361+0000", 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.694e-06 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189366+0000", 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209163+0000", 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01979707 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209236+0000", 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3203999999999999e-05 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210048+0000", 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000811683 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7079 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 1835008~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2626628019999995, 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024715221999999998, 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.512 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7079 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189372+0000", 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189374+0000", 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.052e-06 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189379+0000", 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8950000000000003e-06 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189384+0000", 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211890+0000", 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022506291000000001 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211938+0000", 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7750999999999997e-05 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214086+0000", 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00214818 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7080 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 4456448~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.516 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2626434940000006, 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025413182999999999, 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7080 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:25.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189391+0000", 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189394+0000", 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0230000000000001e-06 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189402+0000", 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189414+0000", 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1639e-05 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214132+0000", 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024717807000000001 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214218+0000", 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6093000000000004e-05 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214804+0000", 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058589500000000001 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7081 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2626012319999997, 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025598254000000001, 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7081 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-05-08T08:25:25.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189436+0000", 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189443+0000", 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189448+0000", 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0810000000000001e-06 2024-05-08T08:25:25.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214836+0000", 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025387742000000001 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214926+0000", 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0665e-05 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215031+0000", 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104678 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.524 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7085 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2625780940000002, 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025822804000000001, 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7085 2024-05-08T08:25:25.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189459+0000", 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3379999999999999e-06 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189465+0000", 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.088e-06 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189470+0000", 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8640000000000004e-06 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215073+0000", 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025603819999999999 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215180+0000", 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106215 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215279+0000", 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9087999999999997e-05 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7086 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2625582529999999, 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026065310000000001, 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:25.529 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7086 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:25.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189478+0000", 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189486+0000", 2024-05-08T08:25:25.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.841e-06 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189491+0000", 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215315+0000", 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025824427 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215411+0000", 2024-05-08T08:25:25.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5606000000000006e-05 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215541+0000", 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012996899999999999 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:25.533 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:25.534 INFO:teuthology.orchestra.run.smithi031.stdout:1592 writesame 0xc742cab thru 0xc7509eb data_size 0x1(0xdd41 bytes) 2024-05-08T08:25:25.535 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1593 read 0x5ce38ff thru 0x5ce7f94 (0x4696 bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1596 writesame 0x90bffea thru 0x90c9602 data_size 0x35(0x9619 bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1597 punch from 0x607a936 to 0x60807f1, (0x5ebb bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1598 read 0x1a11aca thru 0x1a1c7d8 (0xad0f bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1600 read 0x4a16c61 thru 0x4a1734c (0x6ec bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1603 read 0xce8ad98 thru 0xce8c64b (0x18b4 bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1605 write 0x15168b9 thru 0x1518c43 (0x238b bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1606 write 0x8468b60 thru 0x8472a4c (0x9eed bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1608 write 0x6117063 thru 0x611ab00 (0x3a9e bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1609 read 0x70fd8bb thru 0x71048ac (0x6ff2 bytes) 2024-05-08T08:25:25.537 INFO:teuthology.orchestra.run.smithi031.stdout:1610 trunc from 0xec24c1a to 0xcd4d6e8 2024-05-08T08:25:25.538 INFO:teuthology.orchestra.run.smithi031.stdout:1613 write 0x948043e thru 0x9490371 (0xff34 bytes) 2024-05-08T08:25:25.538 INFO:teuthology.orchestra.run.smithi031.stdout:1614 read 0x1b2c61a thru 0x1b311b6 (0x4b9d bytes) 2024-05-08T08:25:25.538 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.538 INFO:teuthology.orchestra.run.smithi031.stdout:1616 write 0xda49678 thru 0xda4ceec (0x3875 bytes) 2024-05-08T08:25:25.538 INFO:teuthology.orchestra.run.smithi031.stdout:1617 trunc from 0xda4ceed to 0x17ba284 2024-05-08T08:25:25.538 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:25.542+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:25.541 INFO:teuthology.orchestra.run.smithi031.stdout:1618 punch from 0x44ec1e to 0x45c487, (0xd869 bytes) 2024-05-08T08:25:25.559 INFO:teuthology.orchestra.run.smithi031.stdout:1619 write 0xbd27328 thru 0xbd2c18d (0x4e66 bytes) 2024-05-08T08:25:25.561 INFO:teuthology.orchestra.run.smithi031.stdout:1620 write 0x22a0400 thru 0x22a31c0 (0x2dc1 bytes) 2024-05-08T08:25:25.564 INFO:teuthology.orchestra.run.smithi031.stdout:1621 read 0xb890fad thru 0xb897402 (0x6456 bytes) 2024-05-08T08:25:25.565 INFO:teuthology.orchestra.run.smithi031.stdout:1623 writesame 0x4c3d2 thru 0x581d9 data_size 0xc(0xbe08 bytes) 2024-05-08T08:25:25.567 INFO:teuthology.orchestra.run.smithi031.stdout:1624 read 0x41ca65e thru 0x41cfed7 (0x587a bytes) 2024-05-08T08:25:25.567 INFO:teuthology.orchestra.run.smithi031.stdout:1625 write 0x4daaead thru 0x4db6bd6 (0xbd2a bytes) 2024-05-08T08:25:25.571 INFO:teuthology.orchestra.run.smithi031.stdout:1626 write 0x3b305a1 thru 0x3b3ee39 (0xe899 bytes) 2024-05-08T08:25:25.574 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.581 INFO:teuthology.orchestra.run.smithi031.stdout:1628 writesame 0xde8a9d3 thru 0xde8cc3d data_size 0xb(0x226b bytes) 2024-05-08T08:25:25.581 INFO:teuthology.orchestra.run.smithi031.stdout:1629 read 0xdca7e8 thru 0xdceb0c (0x4325 bytes) 2024-05-08T08:25:25.582 INFO:teuthology.orchestra.run.smithi031.stdout:1630 punch from 0x486564b to 0x487118b, (0xbb40 bytes) 2024-05-08T08:25:25.588 INFO:teuthology.orchestra.run.smithi031.stdout:1631 trunc from 0xde8cc3e to 0x83abb72 2024-05-08T08:25:25.604 INFO:teuthology.orchestra.run.smithi031.stdout:1632 trunc from 0x83abb72 to 0xdcf80d4 2024-05-08T08:25:25.609 INFO:teuthology.orchestra.run.smithi031.stdout:1634 write 0xb9c98f6 thru 0xb9d5ff5 (0xc700 bytes) 2024-05-08T08:25:25.613 INFO:teuthology.orchestra.run.smithi031.stdout:1635 punch from 0x93b51d6 to 0x93be593, (0x93bd bytes) 2024-05-08T08:25:25.613 INFO:teuthology.orchestra.run.smithi031.stdout:1636 writesame 0xd5a9c88 thru 0xd5b120c data_size 0xb(0x7585 bytes) 2024-05-08T08:25:25.617 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.622 INFO:teuthology.orchestra.run.smithi031.stdout:1638 write 0xe0c01f4 thru 0xe0cd97a (0xd787 bytes) 2024-05-08T08:25:25.625 INFO:teuthology.orchestra.run.smithi031.stdout:1640 write 0x2c6afa1 thru 0x2c6bb1c (0xb7c bytes) 2024-05-08T08:25:25.627 INFO:teuthology.orchestra.run.smithi031.stdout:1641 write 0xa5a103d thru 0xa5a8483 (0x7447 bytes) 2024-05-08T08:25:25.630 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.630 INFO:teuthology.orchestra.run.smithi031.stdout:1643 read 0x792e019 thru 0x79347ee (0x67d6 bytes) 2024-05-08T08:25:25.633 INFO:teuthology.orchestra.run.smithi031.stdout:1644 trunc from 0xe0cd97b to 0xba894c0 2024-05-08T08:25:25.635 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:25.641+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:25.643 INFO:teuthology.orchestra.run.smithi031.stdout:1646 read 0x76cd59 thru 0x7772cc (0xa574 bytes) 2024-05-08T08:25:25.643 INFO:teuthology.orchestra.run.smithi031.stdout:1649 read 0x3497bb thru 0x358413 (0xec59 bytes) 2024-05-08T08:25:25.650 INFO:teuthology.orchestra.run.smithi031.stdout:1650 write 0xd038315 thru 0xd03bd1b (0x3a07 bytes) 2024-05-08T08:25:25.651 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:25.651 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:25.651 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:25.651 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:25.660 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:25.663 INFO:teuthology.orchestra.run.smithi031.stdout:1651 trunc from 0xd03bd1c to 0x489fa8f 2024-05-08T08:25:25.673 INFO:teuthology.orchestra.run.smithi031.stdout:1654 read 0xee2862 thru 0xee7614 (0x4db3 bytes) 2024-05-08T08:25:25.674 INFO:teuthology.orchestra.run.smithi031.stdout:1655 punch from 0x2854e08 to 0x285c3ae, (0x75a6 bytes) 2024-05-08T08:25:25.674 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.674 INFO:teuthology.orchestra.run.smithi031.stdout:1657 punch from 0x25faade to 0x26098e8, (0xee0a bytes) 2024-05-08T08:25:25.690 INFO:teuthology.orchestra.run.smithi031.stdout:1660 write 0xea854af thru 0xea8f7af (0xa301 bytes) 2024-05-08T08:25:25.693 INFO:teuthology.orchestra.run.smithi031.stdout:1661 read 0x110580e thru 0x110e581 (0x8d74 bytes) 2024-05-08T08:25:25.694 INFO:teuthology.orchestra.run.smithi031.stdout:1664 read 0x3aa7cc6 thru 0x3aab77e (0x3ab9 bytes) 2024-05-08T08:25:25.694 INFO:teuthology.orchestra.run.smithi031.stdout:1666 writesame 0x82a370e thru 0x82a70c5 data_size 0x4(0x39b8 bytes) 2024-05-08T08:25:25.697 INFO:teuthology.orchestra.run.smithi031.stdout:1667 punch from 0x3b438b4 to 0x3b4eb02, (0xb24e bytes) 2024-05-08T08:25:25.698 INFO:teuthology.orchestra.run.smithi031.stdout:1670 writesame 0x464aab thru 0x473f4c data_size 0xad(0xf4a2 bytes) 2024-05-08T08:25:25.699 INFO:teuthology.orchestra.run.smithi031.stdout:1671 writesame 0x14b27cb thru 0x14b5141 data_size 0xb(0x2977 bytes) 2024-05-08T08:25:25.702 INFO:teuthology.orchestra.run.smithi031.stdout:1672 punch from 0xa8a7d34 to 0xa8b7212, (0xf4de bytes) 2024-05-08T08:25:25.702 INFO:teuthology.orchestra.run.smithi031.stdout:1673 read 0x52e404e thru 0x52f23dc (0xe38f bytes) 2024-05-08T08:25:25.715 INFO:teuthology.orchestra.run.smithi031.stdout:1674 trunc from 0xea8f7b0 to 0x2a6f947 2024-05-08T08:25:25.729 INFO:teuthology.orchestra.run.smithi031.stdout:1675 read 0xbb9994 thru 0xbbb102 (0x176f bytes) 2024-05-08T08:25:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:1677 read 0x208b0ed thru 0x208e3e6 (0x32fa bytes) 2024-05-08T08:25:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:1678 punch from 0x13c65c6 to 0x13c9318, (0x2d52 bytes) 2024-05-08T08:25:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:1679 read 0x132200c thru 0x132d9bd (0xb9b2 bytes) 2024-05-08T08:25:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:1680 read 0x258251b thru 0x258a44a (0x7f30 bytes) 2024-05-08T08:25:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:1682 punch from 0x294be29 to 0x2956d65, (0xaf3c bytes) 2024-05-08T08:25:25.730 INFO:teuthology.orchestra.run.smithi031.stdout:1684 trunc from 0x2a6f947 to 0x3925a0d 2024-05-08T08:25:25.735 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:25.743+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:25.751 INFO:teuthology.orchestra.run.smithi031.stdout:1686 writesame 0xc54dd97 thru 0xc5558b6 data_size 0x50(0x7b20 bytes) 2024-05-08T08:25:25.751 INFO:teuthology.orchestra.run.smithi031.stdout:1687 punch from 0x681abc1 to 0x68219f4, (0x6e33 bytes) 2024-05-08T08:25:25.751 INFO:teuthology.orchestra.run.smithi031.stdout:1688 write 0x1d0ebe2 thru 0x1d177e9 (0x8c08 bytes) 2024-05-08T08:25:25.755 INFO:teuthology.orchestra.run.smithi031.stdout:1689 read 0x12168d3 thru 0x12168dc (0xa bytes) 2024-05-08T08:25:25.755 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.755 INFO:teuthology.orchestra.run.smithi031.stdout:1694 writesame 0x467edb9 thru 0x468cf50 data_size 0x4(0xe198 bytes) 2024-05-08T08:25:25.762 INFO:teuthology.orchestra.run.smithi031.stdout:1695 read 0x37034d6 thru 0x3706ab0 (0x35db bytes) 2024-05-08T08:25:25.762 INFO:teuthology.orchestra.run.smithi031.stdout:1696 read 0x45ed1e0 thru 0x45eefa0 (0x1dc1 bytes) 2024-05-08T08:25:25.762 INFO:teuthology.orchestra.run.smithi031.stdout:1697 read 0x1751e5d thru 0x175d40c (0xb5b0 bytes) 2024-05-08T08:25:25.762 INFO:teuthology.orchestra.run.smithi031.stdout:1698 read 0xa24610a thru 0xa249a7b (0x3972 bytes) 2024-05-08T08:25:25.763 INFO:teuthology.orchestra.run.smithi031.stdout:1699 trunc from 0xc5558b7 to 0xb193fbe 2024-05-08T08:25:25.769 INFO:teuthology.orchestra.run.smithi031.stdout:1700 read 0xa38f2e5 thru 0xa390106 (0xe22 bytes) 2024-05-08T08:25:25.769 INFO:teuthology.orchestra.run.smithi031.stdout:1701 punch from 0x6d1d2b3 to 0x6d24dde, (0x7b2b bytes) 2024-05-08T08:25:25.769 INFO:teuthology.orchestra.run.smithi031.stdout:1703 read 0x8739c8d thru 0x87467d4 (0xcb48 bytes) 2024-05-08T08:25:25.769 INFO:teuthology.orchestra.run.smithi031.stdout:1704 write 0x95bd0b7 thru 0x95cb045 (0xdf8f bytes) 2024-05-08T08:25:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:1705 punch from 0x1e8706b to 0x1e89f04, (0x2e99 bytes) 2024-05-08T08:25:25.776 INFO:teuthology.orchestra.run.smithi031.stdout:1706 write 0xc508a8 thru 0xc5683c (0x5f95 bytes) 2024-05-08T08:25:25.780 INFO:teuthology.orchestra.run.smithi031.stdout:1709 punch from 0x4321fd3 to 0x4326c56, (0x4c83 bytes) 2024-05-08T08:25:25.780 INFO:teuthology.orchestra.run.smithi031.stdout:1711 read 0x1afa672 thru 0x1aff332 (0x4cc1 bytes) 2024-05-08T08:25:25.782 INFO:teuthology.orchestra.run.smithi031.stdout:1713 trunc from 0xb193fbe to 0x90b9b94 2024-05-08T08:25:25.789 INFO:teuthology.orchestra.run.smithi031.stdout:1714 write 0xc5a4a8 thru 0xc6039f (0x5ef8 bytes) 2024-05-08T08:25:25.791 INFO:teuthology.orchestra.run.smithi031.stdout:1716 read 0x2933db6 thru 0x2942d33 (0xef7e bytes) 2024-05-08T08:25:25.791 INFO:teuthology.orchestra.run.smithi031.stdout:1717 read 0x10178bb thru 0x101ce6c (0x55b2 bytes) 2024-05-08T08:25:25.791 INFO:teuthology.orchestra.run.smithi031.stdout:1718 read 0x53a0e07 thru 0x53affb0 (0xf1aa bytes) 2024-05-08T08:25:25.791 INFO:teuthology.orchestra.run.smithi031.stdout:1719 read 0x449811e thru 0x44a593b (0xd81e bytes) 2024-05-08T08:25:25.792 INFO:teuthology.orchestra.run.smithi031.stdout:1722 punch from 0x71c96b2 to 0x71d1d5c, (0x86aa bytes) 2024-05-08T08:25:25.792 INFO:teuthology.orchestra.run.smithi031.stdout:1723 read 0x5d035b0 thru 0x5d09db8 (0x6809 bytes) 2024-05-08T08:25:25.792 INFO:teuthology.orchestra.run.smithi031.stdout:1724 write 0x30168a3 thru 0x30226fc (0xbe5a bytes) 2024-05-08T08:25:25.795 INFO:teuthology.orchestra.run.smithi031.stdout:1725 write 0x558e469 thru 0x5595254 (0x6dec bytes) 2024-05-08T08:25:25.802 INFO:teuthology.orchestra.run.smithi031.stdout:1726 read 0x3e699ab thru 0x3e71b0e (0x8164 bytes) 2024-05-08T08:25:25.802 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.802 INFO:teuthology.orchestra.run.smithi031.stdout:1729 writesame 0x59f5bdf thru 0x5a02249 data_size 0x1(0xc66b bytes) 2024-05-08T08:25:25.805 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f /var/log/ceph/exp.3.1a.1 2024-05-08T08:25:25.816 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:25.816 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:25.816 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:25.816 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:25.816 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:25.824 INFO:teuthology.orchestra.run.smithi031.stdout:1730 read 0x1b64c31 thru 0x1b7256b (0xd93b bytes) 2024-05-08T08:25:25.826 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:25.831 INFO:teuthology.orchestra.run.smithi031.stdout:1731 trunc from 0x90b9b94 to 0x2a97aa4 2024-05-08T08:25:25.836 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:25.844+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:25.848 INFO:teuthology.orchestra.run.smithi031.stdout:1732 trunc from 0x2a97aa4 to 0x6116d10 2024-05-08T08:25:25.853 INFO:teuthology.orchestra.run.smithi031.stdout:1733 read 0x55adfe1 thru 0x55b5677 (0x7697 bytes) 2024-05-08T08:25:25.867 INFO:teuthology.orchestra.run.smithi031.stdout:1736 write 0xdb82698 thru 0xdb85069 (0x29d2 bytes) 2024-05-08T08:25:25.870 INFO:teuthology.orchestra.run.smithi031.stdout:1737 read 0x6cc02cb thru 0x6ccdc64 (0xd99a bytes) 2024-05-08T08:25:25.870 INFO:teuthology.orchestra.run.smithi031.stdout:1738 write 0x70e52cb thru 0x70f32e5 (0xe01b bytes) 2024-05-08T08:25:25.876 INFO:teuthology.orchestra.run.smithi031.stdout:1739 read 0x892e254 thru 0x893d56a (0xf317 bytes) 2024-05-08T08:25:25.876 INFO:teuthology.orchestra.run.smithi031.stdout:1740 read 0x5e72ba7 thru 0x5e81479 (0xe8d3 bytes) 2024-05-08T08:25:25.876 INFO:teuthology.orchestra.run.smithi031.stdout:1742 writesame 0x2a5ad77 thru 0x2a6a113 data_size 0x1(0xf39d bytes) 2024-05-08T08:25:25.898 INFO:teuthology.orchestra.run.smithi031.stdout:1746 write 0x910a369 thru 0x910d2d2 (0x2f6a bytes) 2024-05-08T08:25:25.904 INFO:teuthology.orchestra.run.smithi031.stdout:1747 read 0x47c5e39 thru 0x47d0b05 (0xaccd bytes) 2024-05-08T08:25:25.904 INFO:teuthology.orchestra.run.smithi031.stdout:1749 writesame 0xd5b7ff2 thru 0xd5b8834 data_size 0x3(0x843 bytes) 2024-05-08T08:25:25.907 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.907 INFO:teuthology.orchestra.run.smithi031.stdout:1751 writesame 0x7124d7f thru 0x7132e3f data_size 0x9(0xe0c1 bytes) 2024-05-08T08:25:25.913 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:25.913 INFO:teuthology.orchestra.run.smithi031.stdout:1757 read 0x1d1a529 thru 0x1d294f8 (0xefd0 bytes) 2024-05-08T08:25:25.922 INFO:teuthology.orchestra.run.smithi031.stdout:1758 write 0xe4ac8e6 thru 0xe4afd58 (0x3473 bytes) 2024-05-08T08:25:25.932 INFO:teuthology.orchestra.run.smithi031.stdout:1762 trunc from 0xe4afd59 to 0x56f0d0d 2024-05-08T08:25:25.937 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:25.944+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:25.945 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:25.945 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:25.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:25.945 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:25.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.945 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.340295236999999, 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098847239999999992, 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 9 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611387+0000", 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.181e-06 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611395+0000", 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0730000000000005e-06 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611411+0000", 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6634999999999999e-05 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611427+0000", 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5699e-05 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611486+0000", 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8692e-05 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611491+0000", 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.989e-06 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615250+0000", 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037591579999999999 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615342+0000", 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2561999999999999e-05 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615480+0000", 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013755399999999999 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.616631+0000", 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011506999999999999 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621178+0000", 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045473620000000001 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621230+0000", 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1845999999999998e-05 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621269+0000", 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9273000000000002e-05 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.951 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1683 3.e e23/18 3:777f8546:::rbd_data.119116cc7dd0.0000000000000187:head v 23'35, mlcod=23'33)", 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.830963642, 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059551170000000002, 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120725+0000", 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3829999999999992e-06 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120734+0000", 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7099999999999996e-06 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120739+0000", 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120789+0000", 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0203e-05 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120806+0000", 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7085999999999999e-05 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126597+0000", 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005790936 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126671+0000", 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3837999999999993e-05 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1680 3.1b e23/18 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head v 23'453, mlcod=23'449)", 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.830901562999999, 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059238520000000003, 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120779+0000", 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120784+0000", 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5809999999999996e-06 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120802+0000", 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8063999999999998e-05 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120809+0000", 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6630000000000004e-06 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120863+0000", 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3918000000000001e-05 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120878+0000", 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4822e-05 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126657+0000", 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057795650000000004 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126702+0000", 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4610000000000001e-05 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1826 3.3 3:c3deaffe:::rbd_data.119116cc7dd0.00000000000001e1:head [set-alloc-hint object_size 262144 write_size 262144,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.700256365, 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0079090730000000008, 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1826 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251425+0000", 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.215e-06 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251433+0000", 2024-05-08T08:25:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9060000000000008e-06 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251444+0000", 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0813e-05 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251449+0000", 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251489+0000", 2024-05-08T08:25:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9774999999999998e-05 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251542+0000", 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3162999999999999e-05 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251630+0000", 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7309e-05 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.252128+0000", 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00049835099999999996 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259274+0000", 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071460289999999999 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259308+0000", 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3679000000000001e-05 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259332+0000", 2024-05-08T08:25:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4752999999999998e-05 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1976 3.3 3:c4f5f4db:::rbd_data.119116cc7dd0.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.543434911, 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065238700000000002, 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1976 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.39e-07 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408247+0000", 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8729999999999999e-06 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408251+0000", 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.357e-06 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408254+0000", 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9699999999999999e-06 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408768+0000", 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00051321699999999995 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408852+0000", 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4888000000000004e-05 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408963+0000", 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110176 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409769+0000", 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00080602600000000005 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414684+0000", 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049152529999999996 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414731+0000", 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.702e-05 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414769+0000", 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7750999999999998e-05 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1972 3.1d e23/18 3:ba47efeb:::rbd_data.119116cc7dd0.0000000000000037:head v 23'32, mlcod=23'30)", 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.543103337, 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0061126660000000001, 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408577+0000", 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3600000000000003e-07 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408583+0000", 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7139999999999998e-06 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408593+0000", 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0429e-05 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408600+0000", 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408870+0000", 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00027024599999999997 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408889+0000", 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8961999999999999e-05 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414654+0000", 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057644200000000001 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414689+0000", 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5537999999999998e-05 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1979 3.1b e23/18 3:d8488a8a:::rbd_data.119116cc7dd0.0000000000000086:head v 23'552, mlcod=23'550)", 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.542699194000001, 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058253389999999997, 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7599999999999997e-07 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408987+0000", 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5149999999999997e-06 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408996+0000", 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.38e-06 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409002+0000", 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1979999999999998e-06 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409049+0000", 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6844000000000001e-05 2024-05-08T08:25:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409067+0000", 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7629999999999999e-05 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414772+0000", 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057050579999999998 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414806+0000", 2024-05-08T08:25:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4137999999999997e-05 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2374 3.1a 3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.230364195, 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011630434, 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2374 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1349999999999999e-06 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721322+0000", 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.13e-06 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721340+0000", 2024-05-08T08:25:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7988000000000001e-05 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721348+0000", 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9789999999999993e-06 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721408+0000", 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0168999999999999e-05 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721494+0000", 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6001000000000006e-05 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721716+0000", 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022242200000000001 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.722577+0000", 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00086067299999999995 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732885+0000", 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010308286 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732916+0000", 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0437999999999999e-05 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732946+0000", 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0213e-05 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7094 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1565839~19667825] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.7464679610000005, 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008628854, 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7094 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205213+0000", 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.54e-07 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206767+0000", 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001554607 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206779+0000", 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1599e-05 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206785+0000", 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0050000000000001e-06 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206831+0000", 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6108e-05 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206916+0000", 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5575000000000007e-05 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207099+0000", 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000182237 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209859+0000", 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027605020000000002 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213776+0000", 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003917238 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213810+0000", 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4051999999999999e-05 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213841+0000", 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0176999999999999e-05 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7095 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.7448811220000007, 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071300790000000001, 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7095 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206801+0000", 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3429999999999998e-06 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206810+0000", 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8389999999999996e-06 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206814+0000", 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1119999999999998e-06 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208857+0000", 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020428310000000002 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208905+0000", 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7849999999999998e-05 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209120+0000", 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000215362 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209862+0000", 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074158700000000004 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213871+0000", 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040092579999999999 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213903+0000", 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1661000000000001e-05 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213929+0000", 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5795e-05 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7096 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.7443741080000006, 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072441390000000001, 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7096 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207309+0000", 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6960000000000001e-06 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207315+0000", 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207319+0000", 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8840000000000004e-06 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209378+0000", 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020587029999999998 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209422+0000", 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4320000000000003e-05 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209632+0000", 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020937899999999999 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.210465+0000", 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083359500000000002 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214459+0000", 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039939270000000004 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214514+0000", 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4338000000000001e-05 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214550+0000", 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6130000000000001e-05 2024-05-08T08:25:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.6803026160000005, 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:25:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:25:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:25:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:25:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:7160 3.1a e28/18 ondisk, result = 0)", 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.6647703329999999, 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0078836489999999995, 2024-05-08T08:25:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286911+0000", 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4580000000000001e-06 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286918+0000", 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9750000000000001e-06 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286935+0000", 2024-05-08T08:25:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7337e-05 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286946+0000", 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0611999999999999e-05 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294654+0000", 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.007707958 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294664+0000", 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0475999999999999e-05 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294793+0000", 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012883299999999999 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:13529 3.14 3:29884244:::rbd_data.119174045df.0000000000000089:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 1.6629637320000001, 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016452866999999999, 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 13529 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288717+0000", 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288721+0000", 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9489999999999998e-06 2024-05-08T08:25:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288730+0000", 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1670000000000005e-06 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288735+0000", 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288772+0000", 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7175000000000002e-05 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288852+0000", 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.291616+0000", 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027647800000000001 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305030+0000", 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013413625 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305098+0000", 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7991999999999995e-05 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305136+0000", 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8197e-05 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305169+0000", 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2601000000000002e-05 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14728 3.14 3:2f5e214b:::rbd_data.119174045df.00000000000000c9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.56757431400000002, 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082457929999999995, 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14728 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384106+0000", 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6300000000000004e-07 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384110+0000", 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2400000000000001e-06 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384121+0000", 2024-05-08T08:25:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0190999999999999e-05 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384126+0000", 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384173+0000", 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299000000000002e-05 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384253+0000", 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9802000000000006e-05 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.385620+0000", 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001366843 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392233+0000", 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066128469999999998 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392250+0000", 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7105999999999999e-05 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392301+0000", 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1453999999999999e-05 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392351+0000", 2024-05-08T08:25:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9883000000000003e-05 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14789 3.9 3:90dc9cea:::rbd_data.119174045df.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.49520456499999999, 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024649007000000001, 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14789 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456476+0000", 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0279999999999999e-06 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456480+0000", 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1280000000000001e-06 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456491+0000", 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0436e-05 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456497+0000", 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.64e-06 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456545+0000", 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7432999999999998e-05 2024-05-08T08:25:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456635+0000", 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9859999999999997e-05 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.460437+0000", 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038026510000000002 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481041+0000", 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020603652 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481053+0000", 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2301e-05 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481088+0000", 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4669999999999998e-05 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481124+0000", 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6208000000000003e-05 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:14789 3.9 e37/18 ondisk, result = 0)", 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.47810015500000003, 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074674299999999997, 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473581+0000", 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.232e-06 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473585+0000", 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473599+0000", 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3859000000000001e-05 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473608+0000", 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7509999999999997e-06 2024-05-08T08:25:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481025+0000", 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074177699999999997 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481037+0000", 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1134e-05 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481047+0000", 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0353000000000001e-05 2024-05-08T08:25:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15094 3.9 3:97b56102:::rbd_data.119174045df.0000000000000061:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.139196613, 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019831228999999999, 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15094 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812484+0000", 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9700000000000005e-07 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812489+0000", 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4559999999999999e-06 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812502+0000", 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2493999999999999e-05 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812509+0000", 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3919999999999997e-06 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812564+0000", 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4565999999999999e-05 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812693+0000", 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012915299999999999 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.816118+0000", 2024-05-08T08:25:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003425329 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832226+0000", 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016107360000000001 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832238+0000", 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2500999999999999e-05 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832279+0000", 2024-05-08T08:25:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0737e-05 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832314+0000", 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5343999999999999e-05 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:15094 3.9 e37/18 ondisk, result = 0)", 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.826002+0000", 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.12567721200000001, 2024-05-08T08:25:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006229058, 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826002+0000", 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826002+0000", 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.039 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826004+0000", 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.336e-06 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826009+0000", 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3029999999999999e-06 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826025+0000", 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5549e-05 2024-05-08T08:25:26.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826035+0000", 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8609999999999993e-06 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832212+0000", 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006177564 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832221+0000", 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7450000000000001e-06 2024-05-08T08:25:26.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832232+0000", 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0699999999999999e-05 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15144 3.3 3:c762a68c:::rbd_data.119174045df.0000000000000027:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.064967670000000005, 2024-05-08T08:25:26.042 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099787780000000006, 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15144 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886713+0000", 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4200000000000005e-07 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.044 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886717+0000", 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4349999999999999e-06 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886728+0000", 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0535e-05 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886735+0000", 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2139999999999999e-06 2024-05-08T08:25:26.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886779+0000", 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4382999999999998e-05 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886827+0000", 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7981e-05 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.888464+0000", 2024-05-08T08:25:26.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001636637 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896530+0000", 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0080662830000000005 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896580+0000", 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9484000000000003e-05 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896642+0000", 2024-05-08T08:25:26.047 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896691+0000", 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8848999999999999e-05 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.048 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:26.050 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:26.051 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:26.056+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1766 writesame 0xe2239af thru 0xe2269d2 data_size 0x34(0x3024 bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1769 read 0x5639de0 thru 0x5640879 (0x6a9a bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1770 read 0x25b90c6 thru 0x25baed5 (0x1e10 bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1772 trunc from 0xe2269d3 to 0x34c9f72 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1774 writesame 0x6dc4fb thru 0x6e93e8 data_size 0x6d(0xceee bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1775 read 0x1d0524d thru 0x1d06378 (0x112c bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1776 punch from 0x28683b1 to 0x286e7ee, (0x643d bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1777 trunc from 0x34c9f72 to 0x20e6a1b 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1779 writesame 0x444825b thru 0x4456dd4 data_size 0x33(0xeb7a bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1781 writesame 0x258f508 thru 0x2598f6f data_size 0xa2(0x9a68 bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1782 write 0x933aa9d thru 0x93430a1 (0x8605 bytes) 2024-05-08T08:25:26.052 INFO:teuthology.orchestra.run.smithi031.stdout:1783 trunc from 0x93430a2 to 0x13e3607 2024-05-08T08:25:26.054 INFO:teuthology.orchestra.run.smithi031.stdout:1784 writesame 0xba8a80d thru 0xba936cd data_size 0x1(0x8ec1 bytes) 2024-05-08T08:25:26.054 INFO:teuthology.orchestra.run.smithi031.stdout:1786 read 0x2dae343 thru 0x2dba460 (0xc11e bytes) 2024-05-08T08:25:26.055 INFO:teuthology.orchestra.run.smithi031.stdout:1787 trunc from 0xba936ce to 0xca162a2 2024-05-08T08:25:26.060 INFO:teuthology.orchestra.run.smithi031.stdout:1790 punch from 0xbeb8b1f to 0xbebc9e8, (0x3ec9 bytes) 2024-05-08T08:25:26.060 INFO:teuthology.orchestra.run.smithi031.stdout:1791 read 0xaf16d08 thru 0xaf24e92 (0xe18b bytes) 2024-05-08T08:25:26.060 INFO:teuthology.orchestra.run.smithi031.stdout:1792 write 0x24121e5 thru 0x2420f86 (0xeda2 bytes) 2024-05-08T08:25:26.066 INFO:teuthology.orchestra.run.smithi031.stdout:1793 punch from 0x827ad5b to 0x8288632, (0xd8d7 bytes) 2024-05-08T08:25:26.066 INFO:teuthology.orchestra.run.smithi031.stdout:1795 read 0x155ac95 thru 0x1566395 (0xb701 bytes) 2024-05-08T08:25:26.066 INFO:teuthology.orchestra.run.smithi031.stdout:1801 punch from 0x2526bb to 0x25a13a, (0x7a7f bytes) 2024-05-08T08:25:26.066 INFO:teuthology.orchestra.run.smithi031.stdout:1802 writesame 0x8866a1a thru 0x886c538 data_size 0x1(0x5b1f bytes) 2024-05-08T08:25:26.086 INFO:teuthology.orchestra.run.smithi031.stdout:1803 write 0xe1ceb68 thru 0xe1d0a1c (0x1eb5 bytes) 2024-05-08T08:25:26.089 INFO:teuthology.orchestra.run.smithi031.stdout:1804 writesame 0x22dedf7 thru 0x22ec692 data_size 0x6(0xd89c bytes) 2024-05-08T08:25:26.095 INFO:teuthology.orchestra.run.smithi031.stdout:1805 write 0x901bd5a thru 0x902840d (0xc6b4 bytes) 2024-05-08T08:25:26.099 INFO:teuthology.orchestra.run.smithi031.stdout:1806 write 0x1e30088 thru 0x1e3f5f8 (0xf571 bytes) 2024-05-08T08:25:26.102 INFO:teuthology.orchestra.run.smithi031.stdout:1808 write 0x2ccbaa6 thru 0x2cd5e7a (0xa3d5 bytes) 2024-05-08T08:25:26.106 INFO:teuthology.orchestra.run.smithi031.stdout:1809 clone 11 order 19 su 4096 sc 7 2024-05-08T08:25:26.150 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:26.158+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:26.197 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:26.197 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:26.197 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:26.198 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:26.206 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:26.251 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:26.258+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:26.298 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:26.312 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:26.312 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:26.312 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:26.312 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:26.312 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:26.321 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:26.362 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:26.369+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:26.427 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:26.427 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:26.445 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:26.445 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:26.453 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:26.460+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:26.462 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:26.462 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:26.469 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:26.469 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:26.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:26.469 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1700 3.1d 3:bbece77f:::rbd_data.119116cc7dd0.0000000000000263:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.336000491, 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049688420000000002, 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1700 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3900000000000004e-07 2024-05-08T08:25:26.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139611+0000", 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7419999999999999e-06 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139621+0000", 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6369999999999994e-06 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139636+0000", 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5367e-05 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139684+0000", 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899000000000003e-05 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139733+0000", 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9205000000000001e-05 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:26.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139798+0000", 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4431999999999996e-05 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144321+0000", 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045238159999999999 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144511+0000", 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018977100000000001 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144544+0000", 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2620000000000003e-05 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144575+0000", 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1714000000000002e-05 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1972 3.1d 3:ba47efeb:::rbd_data.119116cc7dd0.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:26.475 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.067358171999999, 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066697570000000001, 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1972 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408250+0000", 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.026e-06 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408256+0000", 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8490000000000002e-06 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408268+0000", 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1827999999999999e-05 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408275+0000", 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408331+0000", 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5989000000000002e-05 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408406+0000", 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5666000000000004e-05 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408509+0000", 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010266400000000001 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.410688+0000", 2024-05-08T08:25:26.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002179112 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414827+0000", 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041387699999999999 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414878+0000", 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1480999999999997e-05 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414919+0000", 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0188999999999998e-05 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1989 3.1c 3:39f766e8:::rbd_data.119116cc7dd0.0000000000000145:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.063231165000001, 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005150277, 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.481 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1989 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412377+0000", 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3600000000000003e-07 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412380+0000", 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7639999999999999e-06 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412390+0000", 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9729999999999992e-06 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412396+0000", 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2519999999999999e-06 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412447+0000", 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1177999999999997e-05 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412525+0000", 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8375999999999996e-05 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412612+0000", 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7106000000000003e-05 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415079+0000", 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024662569999999999 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417466+0000", 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023874170000000002 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417498+0000", 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2215000000000002e-05 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417526+0000", 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8002999999999999e-05 2024-05-08T08:25:26.486 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1994 3.5 3:a2dbaef4:::rbd_data.119116cc7dd0.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.060375096000001, 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052774390000000001, 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.487 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1994 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415233+0000", 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.61e-07 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415238+0000", 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7309999999999998e-06 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415250+0000", 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.118e-05 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415256+0000", 2024-05-08T08:25:26.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.122e-06 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415308+0000", 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2111000000000003e-05 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415389+0000", 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0889000000000004e-05 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:26.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415492+0000", 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000103316 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416315+0000", 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000822804 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420358+0000", 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004043128 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420470+0000", 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000112197 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420509+0000", 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9199999999999997e-05 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1995 3.17 e23/18 3:eb8d97a8:::rbd_data.119116cc7dd0.0000000000000175:head v 23'20, mlcod=23'18)", 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.059871242, 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072714820000000001, 2024-05-08T08:25:26.492 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9200000000000001e-07 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415742+0000", 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415751+0000", 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2890000000000003e-06 2024-05-08T08:25:26.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415757+0000", 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0190000000000003e-06 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415804+0000", 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7141999999999997e-05 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415824+0000", 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0136000000000002e-05 2024-05-08T08:25:26.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.422967+0000", 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071422839999999996 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423007+0000", 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0720000000000003e-05 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2002 3.1f e23/18 3:fbb43922:::rbd_data.119116cc7dd0.00000000000001e2:head v 23'25, mlcod=23'23)", 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.057715963, 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051483420000000002, 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417892+0000", 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417896+0000", 2024-05-08T08:25:26.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9339999999999999e-06 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417905+0000", 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8379999999999997e-06 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417914+0000", 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9509999999999995e-06 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417969+0000", 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4225999999999999e-05 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417988+0000", 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9318999999999999e-05 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423010+0000", 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050221659999999998 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423039+0000", 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9417999999999999e-05 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2008 3.5 3:a3d8901a:::rbd_data.119116cc7dd0.0000000000000254:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.420846+0000", 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.054761281000001, 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049252009999999997, 2024-05-08T08:25:26.501 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2008 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420846+0000", 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420846+0000", 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420847+0000", 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420851+0000", 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0330000000000002e-06 2024-05-08T08:25:26.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420861+0000", 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9799999999999993e-06 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420866+0000", 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0950000000000003e-06 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420926+0000", 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0590000000000001e-05 2024-05-08T08:25:26.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421006+0000", 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.928e-05 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421099+0000", 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3850000000000004e-05 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421938+0000", 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083869299999999995 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425680+0000", 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037421989999999999 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425731+0000", 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1073999999999999e-05 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425771+0000", 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9669000000000003e-05 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2011 3.16 3:6eef803b:::rbd_data.119116cc7dd0.0000000000000257:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.054655575000002, 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050707879999999997, 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2011 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1299999999999998e-07 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420957+0000", 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170000000000001e-06 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420969+0000", 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1938e-05 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420976+0000", 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3200000000000002e-06 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421035+0000", 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8696999999999997e-05 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421080+0000", 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4709000000000002e-05 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421139+0000", 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9762999999999997e-05 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425663+0000", 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045239520000000004 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425971+0000", 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00030778899999999999 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425999+0000", 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8005e-05 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.426022+0000", 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3184999999999999e-05 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2126 3.16 3:6a294600:::rbd_data.119116cc7dd0.000000000000015b:head [set-alloc-hint object_size 262144 write_size 262144,write 59392~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.954709635, 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066558700000000004, 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2126 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520898+0000", 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7999999999999993e-07 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520908+0000", 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2520000000000005e-06 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520920+0000", 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1891e-05 2024-05-08T08:25:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520927+0000", 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4950000000000002e-06 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520980+0000", 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2895000000000001e-05 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521065+0000", 2024-05-08T08:25:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4698999999999997e-05 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521193+0000", 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012812199999999999 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521826+0000", 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063310000000000005 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527495+0000", 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056689669999999996 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527529+0000", 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4394000000000003e-05 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527553+0000", 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4074999999999999e-05 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2133 3.16 3:6a294600:::rbd_data.119116cc7dd0.000000000000015b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.935396466, 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065993600000000003, 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2133 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7800000000000001e-07 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540215+0000", 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7689999999999998e-06 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540225+0000", 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0039e-05 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540230+0000", 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2830000000000001e-06 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540280+0000", 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9480000000000001e-05 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540352+0000", 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2347000000000004e-05 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540409+0000", 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6518999999999999e-05 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541006+0000", 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00059697199999999998 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546717+0000", 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057116579999999997 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546764+0000", 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7039000000000001e-05 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546810+0000", 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5575999999999999e-05 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2140 3.5 3:a1b38341:::rbd_data.119116cc7dd0.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.933755103999999, 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00497986, 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2140 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541853+0000", 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:25:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541857+0000", 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5469999999999998e-06 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541868+0000", 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0505e-05 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541873+0000", 2024-05-08T08:25:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0989999999999998e-06 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541923+0000", 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0383000000000002e-05 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542000+0000", 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6960999999999997e-05 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542106+0000", 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000105717 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542543+0000", 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00043726399999999998 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042138540000000004 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546793+0000", 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6269000000000001e-05 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546832+0000", 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8464000000000003e-05 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2145 3.1d 3:bdb88d54:::rbd_data.119116cc7dd0.0000000000000326:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.928850564999999, 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006666384, 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2145 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2e-07 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546760+0000", 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8990000000000001e-06 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546766+0000", 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5390000000000004e-06 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546773+0000", 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.19e-06 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547312+0000", 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053938900000000004 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547380+0000", 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8189999999999996e-05 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547474+0000", 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3450999999999997e-05 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.550283+0000", 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002809228 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553330+0000", 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030471249999999999 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553378+0000", 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7710000000000002e-05 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553423+0000", 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5142999999999998e-05 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2154 3.d e23/18 3:b40d5f62:::rbd_data.119116cc7dd0.000000000000039b:head v 23'28, mlcod=23'26)", 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.548514+0000", 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.927093074, 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048007409999999999, 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548514+0000", 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548514+0000", 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548515+0000", 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999997e-07 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548520+0000", 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2270000000000001e-06 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548531+0000", 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0763e-05 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548537+0000", 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6270000000000002e-06 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548579+0000", 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.125e-05 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.548592+0000", 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3553999999999999e-05 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553275+0000", 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0046825360000000002 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553315+0000", 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0145999999999998e-05 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2197 3.1d 3:be511a0a:::rbd_data.119116cc7dd0.0000000000000209:head [set-alloc-hint object_size 262144 write_size 262144,write 4096~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.885496570000001, 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004962401, 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2197 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590112+0000", 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0419999999999999e-06 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590120+0000", 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5050000000000007e-06 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590131+0000", 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.077e-05 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590136+0000", 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590180+0000", 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3299e-05 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590259+0000", 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8848000000000001e-05 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590343+0000", 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4328999999999998e-05 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.592143+0000", 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001800248 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595017+0000", 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002873553 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595049+0000", 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2481999999999998e-05 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595073+0000", 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.37e-05 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2338 3.16 3:6f361ecb:::rbd_data.119116cc7dd0.000000000000001b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.768089776, 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052194750000000003, 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2338 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707518+0000", 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707525+0000", 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7900000000000002e-06 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707549+0000", 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3414999999999999e-05 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707558+0000", 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1600000000000004e-06 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707639+0000", 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0772000000000005e-05 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707722+0000", 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3366000000000001e-05 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707911+0000", 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000189447 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.709463+0000", 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001551621 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712639+0000", 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031764419999999998 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712693+0000", 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3402000000000001e-05 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712737+0000", 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3897999999999999e-05 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2713 3.2 e24/18 3:42c14b71:::rbd_data.1191a2472683.0000000000000000:head v 24'16, mlcod=23'14)", 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.442319936000001, 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058213900000000001, 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033289+0000", 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036196+0000", 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0029067730000000001 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036213+0000", 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7394e-05 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036228+0000", 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4841000000000001e-05 2024-05-08T08:25:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036289+0000", 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1091000000000002e-05 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036310+0000", 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0951e-05 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.039072+0000", 2024-05-08T08:25:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002761763 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.039109+0000", 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6656999999999998e-05 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2782 3.1c 3:3afc57c8:::rbd_data.1191a2472683.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23855104~19711] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.418878529000001, 2024-05-08T08:25:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010424643000000001, 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2782 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056730+0000", 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9699999999999994e-07 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061776+0000", 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050462679999999996 2024-05-08T08:25:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061789+0000", 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3358e-05 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061795+0000", 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1349999999999997e-06 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061844+0000", 2024-05-08T08:25:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8414000000000001e-05 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061957+0000", 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000112759 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.062210+0000", 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000253918 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.066162+0000", 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039516179999999996 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067075+0000", 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00091328600000000002 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067116+0000", 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0599999999999998e-05 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067153+0000", 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7289999999999997e-05 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14439 3.5 3:a4925d73:::rbd_data.119174045df.000000000000001d:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.079083+0000", 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 1.3965239549999999, 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004841166, 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14439 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079083+0000", 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079083+0000", 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079084+0000", 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5799999999999998e-07 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079092+0000", 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9019999999999999e-06 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079102+0000", 2024-05-08T08:25:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7010000000000008e-06 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079107+0000", 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1499999999999998e-06 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079176+0000", 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9566000000000004e-05 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.079261+0000", 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4816999999999998e-05 2024-05-08T08:25:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.080154+0000", 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00089261699999999995 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.083841+0000", 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036869939999999999 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.083869+0000", 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7985999999999999e-05 2024-05-08T08:25:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.083899+0000", 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0642999999999998e-05 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.083924+0000", 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4932000000000001e-05 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14729 3.1c 3:3a8d2cd7:::rbd_data.119174045df.00000000000000ca:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.384288+0000", 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 1.0913194909999999, 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018281189, 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14729 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384288+0000", 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384288+0000", 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384289+0000", 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1699999999999997e-07 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384294+0000", 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3199999999999999e-06 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384306+0000", 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2296999999999999e-05 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384313+0000", 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1910000000000003e-06 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384359+0000", 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5759e-05 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384448+0000", 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8547000000000005e-05 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.387555+0000", 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003107014 2024-05-08T08:25:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402424+0000", 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014869413999999999 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402495+0000", 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0641000000000003e-05 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402534+0000", 2024-05-08T08:25:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9387000000000001e-05 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402569+0000", 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4702000000000001e-05 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:14729 3.1c e37/18 ondisk, result = 0)", 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 1.078343568, 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053112380000000002, 2024-05-08T08:25:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7000000000000003e-07 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397268+0000", 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.743e-06 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397278+0000", 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0231000000000001e-05 2024-05-08T08:25:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397284+0000", 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7459999999999997e-06 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402483+0000", 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051991859999999997 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402491+0000", 2024-05-08T08:25:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402575+0000", 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3938999999999997e-05 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:26.576 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:26.577 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:26.578 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:26.578 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:26.583 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:26.583 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:26.584 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:26.584 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:26.584 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:26.585 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:26.585 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:26.585 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:26.586 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:26.585+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:26.678 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:26.685+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:26.692 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:26.692 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:26.701 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:26.785 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:26.792+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:26.846 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:26.846 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:26.855 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:26.880 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:26.888+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:26.981 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:26.988+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:26.999 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:26.999 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:27.008 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:27.038 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:25:27.082 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:27.088+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:27.119 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone10 2024-05-08T08:25:27.122 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:27.122 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:27.131 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:27.215 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:27.222+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:27.263 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-05-08T08:25:27.279 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:27.280 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:27.284 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:27.291+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:27.289 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:27.385 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:27.392+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:27.437 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:27.437 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:27.445 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:27.486 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:27.493+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:27.493 INFO:teuthology.orchestra.run.smithi031.stdout:1810 write 0x9ce977 thru 0x9d3f96 (0x5620 bytes) 2024-05-08T08:25:27.542 INFO:teuthology.orchestra.run.smithi031.stdout:1812 write 0x20fda13 thru 0x2109edd (0xc4cb bytes) 2024-05-08T08:25:27.562 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.562 INFO:teuthology.orchestra.run.smithi031.stdout:1821 read 0x2dcefb1 thru 0x2ddafd6 (0xc026 bytes) 2024-05-08T08:25:27.562 INFO:teuthology.orchestra.run.smithi031.stdout:1824 read 0x2a9c486 thru 0x2a9d134 (0xcaf bytes) 2024-05-08T08:25:27.562 INFO:teuthology.orchestra.run.smithi031.stdout:1825 read 0xdbdac88 thru 0xdbe606a (0xb3e3 bytes) 2024-05-08T08:25:27.562 INFO:teuthology.orchestra.run.smithi031.stdout:1827 writesame 0x14c33f2 thru 0x14ca5cc data_size 0x1(0x71db bytes) 2024-05-08T08:25:27.586 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:27.593+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:27.588 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.589 INFO:teuthology.orchestra.run.smithi031.stdout:1833 writesame 0x4557db5 thru 0x455db30 data_size 0x1f(0x5d7c bytes) 2024-05-08T08:25:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:1834 punch from 0xbc4de92 to 0xbc56107, (0x8275 bytes) 2024-05-08T08:25:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:1836 punch from 0x8b29242 to 0x8b34409, (0xb1c7 bytes) 2024-05-08T08:25:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:1838 punch from 0x139c383 to 0x13a5941, (0x95be bytes) 2024-05-08T08:25:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:1839 read 0x56ec729 thru 0x56f1717 (0x4fef bytes) 2024-05-08T08:25:27.598 INFO:teuthology.orchestra.run.smithi031.stdout:1841 writesame 0xce72bc thru 0xcf6fc9 data_size 0x12(0xfd0e bytes) 2024-05-08T08:25:27.599 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:1843 read 0xda02370 thru 0xda08b78 (0x6809 bytes) 2024-05-08T08:25:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.624 INFO:teuthology.orchestra.run.smithi031.stdout:1847 writesame 0xfa165f thru 0xfb0310 data_size 0x3(0xecb2 bytes) 2024-05-08T08:25:27.659 INFO:teuthology.orchestra.run.smithi031.stdout:1848 trunc from 0xe1d0a1d to 0xa17b008 2024-05-08T08:25:27.674 INFO:teuthology.orchestra.run.smithi031.stdout:1849 write 0x3cd1b8b thru 0x3cda450 (0x88c6 bytes) 2024-05-08T08:25:27.687 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:27.694+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:27.688 INFO:teuthology.orchestra.run.smithi031.stdout:1851 write 0x85ab0c9 thru 0x85ab210 (0x148 bytes) 2024-05-08T08:25:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:1852 punch from 0x687cd5d to 0x688a9b1, (0xdc54 bytes) 2024-05-08T08:25:27.694 INFO:teuthology.orchestra.run.smithi031.stdout:1853 write 0x3646182 thru 0x3653cd9 (0xdb58 bytes) 2024-05-08T08:25:27.713 INFO:teuthology.orchestra.run.smithi031.stdout:1855 write 0x1414a4 thru 0x1447ba (0x3317 bytes) 2024-05-08T08:25:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:27.745 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:27.756 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:27.756 INFO:teuthology.orchestra.run.smithi031.stdout:1856 write 0xca409fd thru 0xca4b71e (0xad22 bytes) 2024-05-08T08:25:27.771 INFO:teuthology.orchestra.run.smithi031.stdout:1857 trunc from 0xca4b71f to 0x917d273 2024-05-08T08:25:27.810 INFO:teuthology.orchestra.run.smithi031.stdout:1858 write 0xae6d8f2 thru 0xae6de51 (0x560 bytes) 2024-05-08T08:25:27.811 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:27.818+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:27.818 INFO:teuthology.orchestra.run.smithi031.stdout:1861 write 0x9537296 thru 0x954641a (0xf185 bytes) 2024-05-08T08:25:27.828 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.828 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.828 INFO:teuthology.orchestra.run.smithi031.stdout:1865 write 0x55a1720 thru 0x55a478c (0x306d bytes) 2024-05-08T08:25:27.838 INFO:teuthology.orchestra.run.smithi031.stdout:1866 write 0x3823d51 thru 0x3824684 (0x934 bytes) 2024-05-08T08:25:27.848 INFO:teuthology.orchestra.run.smithi031.stdout:1867 trunc from 0xae6de52 to 0x789c025 2024-05-08T08:25:27.873 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.873 INFO:teuthology.orchestra.run.smithi031.stdout:1869 read 0x58a4902 thru 0x58a65d6 (0x1cd5 bytes) 2024-05-08T08:25:27.885 INFO:teuthology.orchestra.run.smithi031.stdout:1870 write 0xcb6b4b6 thru 0xcb73314 (0x7e5f bytes) 2024-05-08T08:25:27.890 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:27.896+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:27.891 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:27.891 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:1871 punch from 0x9c4b8e to 0x9cfec7, (0xb339 bytes) 2024-05-08T08:25:27.900 INFO:teuthology.orchestra.run.smithi031.stdout:1875 read 0x3923937 thru 0x3926694 (0x2d5e bytes) 2024-05-08T08:25:27.901 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:27.908 INFO:teuthology.orchestra.run.smithi031.stdout:1876 write 0xe13c499 thru 0xe14bab7 (0xf61f bytes) 2024-05-08T08:25:27.916 INFO:teuthology.orchestra.run.smithi031.stdout:1877 read 0x5e6b30 thru 0x5f5c66 (0xf137 bytes) 2024-05-08T08:25:27.932 INFO:teuthology.orchestra.run.smithi031.stdout:1878 trunc from 0xe14bab8 to 0x1ed07c2 2024-05-08T08:25:27.984 INFO:teuthology.orchestra.run.smithi031.stdout:1879 trunc from 0x1ed07c2 to 0x1193409 2024-05-08T08:25:27.991 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:27.998+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:27.999 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.999 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:27.999 INFO:teuthology.orchestra.run.smithi031.stdout:1883 read 0x6348cf thru 0x6433bb (0xeaed bytes) 2024-05-08T08:25:28.016 INFO:teuthology.orchestra.run.smithi031.stdout:1885 write 0x74dbdbf thru 0x74dfa72 (0x3cb4 bytes) 2024-05-08T08:25:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:1888 read 0x4b488aa thru 0x4b5314a (0xa8a1 bytes) 2024-05-08T08:25:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:1889 read 0x47842b0 thru 0x479305b (0xedac bytes) 2024-05-08T08:25:28.026 INFO:teuthology.orchestra.run.smithi031.stdout:1890 writesame 0x662d781 thru 0x663bdb7 data_size 0x5(0xe637 bytes) 2024-05-08T08:25:28.037 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:28.037 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:28.038 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.038 INFO:teuthology.orchestra.run.smithi031.stdout:1893 write 0xd358b1 thru 0xd3e3ee (0x8b3e bytes) 2024-05-08T08:25:28.039 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:25:28.045 INFO:teuthology.orchestra.run.smithi031.stdout:1894 write 0xa46fd4 thru 0xa54582 (0xd5af bytes) 2024-05-08T08:25:28.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:28.057 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.057 INFO:teuthology.orchestra.run.smithi031.stdout:1898 read 0x5662994 thru 0x5670549 (0xdbb6 bytes) 2024-05-08T08:25:28.058 INFO:teuthology.orchestra.run.smithi031.stdout:1900 trunc from 0x74dfa73 to 0x64eafed 2024-05-08T08:25:28.082 INFO:teuthology.orchestra.run.smithi031.stdout:1901 writesame 0xbfe76d6 thru 0xbfe7ef3 data_size 0x1(0x81e bytes) 2024-05-08T08:25:28.082 INFO:teuthology.orchestra.run.smithi031.stdout:1904 read 0x783badd thru 0x7845405 (0x9929 bytes) 2024-05-08T08:25:28.088 INFO:teuthology.orchestra.run.smithi031.stdout:1905 write 0xcb7c743 thru 0xcb82d04 (0x65c2 bytes) 2024-05-08T08:25:28.091 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:28.098+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:1909 read 0xb69e8a8 thru 0xb6ac552 (0xdcab bytes) 2024-05-08T08:25:28.101 INFO:teuthology.orchestra.run.smithi031.stdout:1910 writesame 0x809fa78 thru 0x80add2d data_size 0x33(0xe2b6 bytes) 2024-05-08T08:25:28.110 INFO:teuthology.orchestra.run.smithi031.stdout:1911 punch from 0x890500b to 0x89108cd, (0xb8c2 bytes) 2024-05-08T08:25:28.124 INFO:teuthology.orchestra.run.smithi031.stdout:1913 trunc from 0xcb82d05 to 0x633895 2024-05-08T08:25:28.192 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:28.198+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:28.195 INFO:teuthology.orchestra.run.smithi031.stdout:1914 read 0x58d156 thru 0x59c72b (0xf5d6 bytes) 2024-05-08T08:25:28.196 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:28.196 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:28.213 INFO:teuthology.orchestra.run.smithi031.stdout:1915 trunc from 0x633895 to 0xcf3c5c9 2024-05-08T08:25:28.219 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.219 INFO:teuthology.orchestra.run.smithi031.stdout:1917 read 0x7dbb111 thru 0x7dcb06a (0xff5a bytes) 2024-05-08T08:25:28.220 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:28.220 INFO:teuthology.orchestra.run.smithi031.stdout:1918 write 0x82eff6d thru 0x82f0da7 (0xe3b bytes) 2024-05-08T08:25:28.222 INFO:teuthology.orchestra.run.smithi031.stdout:1920 write 0x89715a3 thru 0x8973e87 (0x28e5 bytes) 2024-05-08T08:25:28.227 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.227 INFO:teuthology.orchestra.run.smithi031.stdout:1923 write 0x6a377b6 thru 0x6a3afbc (0x3807 bytes) 2024-05-08T08:25:28.233 INFO:teuthology.orchestra.run.smithi031.stdout:1924 read 0xcf1c8c4 thru 0xcf278ac (0xafe9 bytes) 2024-05-08T08:25:28.233 INFO:teuthology.orchestra.run.smithi031.stdout:1925 punch from 0xc59f7a4 to 0xc5a9ac4, (0xa320 bytes) 2024-05-08T08:25:28.233 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.233 INFO:teuthology.orchestra.run.smithi031.stdout:1927 punch from 0xb5bf0b3 to 0xb5c58d8, (0x6825 bytes) 2024-05-08T08:25:28.233 INFO:teuthology.orchestra.run.smithi031.stdout:1928 punch from 0x1f9f143 to 0x1fac88d, (0xd74a bytes) 2024-05-08T08:25:28.234 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.234 INFO:teuthology.orchestra.run.smithi031.stdout:1931 write 0x36e379e thru 0x36ee683 (0xaee6 bytes) 2024-05-08T08:25:28.244 INFO:teuthology.orchestra.run.smithi031.stdout:1932 write 0x7ba88e6 thru 0x7baedda (0x64f5 bytes) 2024-05-08T08:25:28.254 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.254 INFO:teuthology.orchestra.run.smithi031.stdout:1934 write 0x90fa34e thru 0x91097c1 (0xf474 bytes) 2024-05-08T08:25:28.263 INFO:teuthology.orchestra.run.smithi031.stdout:1935 trunc from 0xcf3c5c9 to 0xc63ae07 2024-05-08T08:25:28.269 INFO:teuthology.orchestra.run.smithi031.stdout:1937 write 0xbe25d68 thru 0xbe2a827 (0x4ac0 bytes) 2024-05-08T08:25:28.279 INFO:teuthology.orchestra.run.smithi031.stdout:1938 write 0x6d7a14e thru 0x6d805da (0x648d bytes) 2024-05-08T08:25:28.292 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:28.299+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:28.296 INFO:teuthology.orchestra.run.smithi031.stdout:1939 trunc from 0xc63ae07 to 0x655fa9a 2024-05-08T08:25:28.315 INFO:teuthology.orchestra.run.smithi031.stdout:1940 read 0x28dafff thru 0x28e0dfb (0x5dfd bytes) 2024-05-08T08:25:28.315 INFO:teuthology.orchestra.run.smithi031.stdout:1941 writesame 0x1eefed7 thru 0x1efec64 data_size 0xd(0xed8e bytes) 2024-05-08T08:25:28.324 INFO:teuthology.orchestra.run.smithi031.stdout:1942 punch from 0xb781ab to 0xb8630e, (0xe163 bytes) 2024-05-08T08:25:28.324 INFO:teuthology.orchestra.run.smithi031.stdout:1943 read 0x504029d thru 0x504bf12 (0xbc76 bytes) 2024-05-08T08:25:28.324 INFO:teuthology.orchestra.run.smithi031.stdout:1944 writesame 0x17ca3af thru 0x17d2309 data_size 0x1(0x7f5b bytes) 2024-05-08T08:25:28.344 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:28.344 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:28.346 INFO:teuthology.orchestra.run.smithi031.stdout:1945 write 0x7f594c5 thru 0x7f65680 (0xc1bc bytes) 2024-05-08T08:25:28.354 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:28.358 INFO:teuthology.orchestra.run.smithi031.stdout:1946 punch from 0x2d756f8 to 0x2d80044, (0xa94c bytes) 2024-05-08T08:25:28.358 INFO:teuthology.orchestra.run.smithi031.stdout:1947 punch from 0x53c21cc to 0x53c93d7, (0x720b bytes) 2024-05-08T08:25:28.358 INFO:teuthology.orchestra.run.smithi031.stdout:1949 write 0x1f4c79f thru 0x1f5c144 (0xf9a6 bytes) 2024-05-08T08:25:28.366 INFO:teuthology.orchestra.run.smithi031.stdout:1950 write 0x81dcace thru 0x81df54b (0x2a7e bytes) 2024-05-08T08:25:28.376 INFO:teuthology.orchestra.run.smithi031.stdout:1951 punch from 0x6f12c02 to 0x6f20aa3, (0xdea1 bytes) 2024-05-08T08:25:28.376 INFO:teuthology.orchestra.run.smithi031.stdout:1952 writesame 0x3ea1023 thru 0x3ea6718 data_size 0x1(0x56f6 bytes) 2024-05-08T08:25:28.389 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:28.389 INFO:teuthology.orchestra.run.smithi031.stdout:1956 read 0x3551727 thru 0x35549c7 (0x32a1 bytes) 2024-05-08T08:25:28.390 INFO:teuthology.orchestra.run.smithi031.stdout:1957 write 0x417d49a thru 0x418761a (0xa181 bytes) 2024-05-08T08:25:28.393 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:28.399+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:28.406 INFO:teuthology.orchestra.run.smithi031.stdout:1958 read 0x7088f49 thru 0x70947de (0xb896 bytes) 2024-05-08T08:25:28.407 INFO:teuthology.orchestra.run.smithi031.stdout:1959 trunc from 0x81df54c to 0x8fc63bd 2024-05-08T08:25:28.413 INFO:teuthology.orchestra.run.smithi031.stdout:1962 read 0x3f88a59 thru 0x3f93870 (0xae18 bytes) 2024-05-08T08:25:28.413 INFO:teuthology.orchestra.run.smithi031.stdout:1963 clone 12 order 25 su 262144 sc 2 2024-05-08T08:25:28.470 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:28.471 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:28.479 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:28.521 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:28.528+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:28.587 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:28.594 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:28.601+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:28.631 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:28.631 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:28.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:28.695 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:28.703+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:28.716 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:28.716 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:28.716 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:28.735 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:28.735 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:28.752 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:28.752 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:28.753 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:28.753 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:28.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:28.753 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:28.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6425 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.051412184, 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03287702, 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:28.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6425 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708825+0000", 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7e-06 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708833+0000", 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708851+0000", 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7961e-05 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708859+0000", 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2439999999999993e-06 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708920+0000", 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1178000000000003e-05 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.709031+0000", 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011102 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.714353+0000", 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053222440000000003 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741561+0000", 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027207346 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741581+0000", 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0854000000000001e-05 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741644+0000", 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2526000000000004e-05 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741700+0000", 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5812e-05 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6447 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.985978434, 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020412153999999998, 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6447 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774258+0000", 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774267+0000", 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8869999999999998e-06 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774285+0000", 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7399000000000001e-05 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774293+0000", 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1089999999999998e-06 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774350+0000", 2024-05-08T08:25:28.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774416+0000", 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6166000000000003e-05 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.777145+0000", 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027283160000000002 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794536+0000", 2024-05-08T08:25:28.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017391163000000001 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794555+0000", 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9377000000000001e-05 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794614+0000", 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8829000000000002e-05 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794669+0000", 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:25:28.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7045 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 2883584~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188874+0000", 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571360653999999, 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019434, 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7045 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.765 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188874+0000", 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188874+0000", 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188875+0000", 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:25:28.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188877+0000", 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188883+0000", 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2210000000000002e-06 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188895+0000", 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1970000000000001e-05 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207453+0000", 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018558270000000002 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207822+0000", 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00036873199999999998 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208308+0000", 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048613299999999998 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7046 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 5505024~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571332709, 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020079657000000001, 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7046 2024-05-08T08:25:28.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188903+0000", 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:25:28.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188905+0000", 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188911+0000", 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188915+0000", 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208342+0000", 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019426335999999999 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208397+0000", 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5417000000000002e-05 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208982+0000", 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058485500000000001 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7047 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:28.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571313832, 2024-05-08T08:25:28.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020213064999999999, 2024-05-08T08:25:28.773 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7047 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2099999999999998e-07 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188923+0000", 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188929+0000", 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188934+0000", 2024-05-08T08:25:28.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7360000000000001e-06 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209012+0000", 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020078822999999999 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209078+0000", 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5859999999999996e-05 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209134+0000", 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5952999999999998e-05 2024-05-08T08:25:28.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7051 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571294802000001, 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020448772, 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7051 2024-05-08T08:25:28.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188941+0000", 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2800000000000003e-07 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188943+0000", 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188949+0000", 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7819999999999999e-06 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188954+0000", 2024-05-08T08:25:28.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.164e-06 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209169+0000", 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020215151000000001 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209252+0000", 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3341e-05 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209389+0000", 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000136704 2024-05-08T08:25:28.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7052 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571275644, 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020652770000000001, 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7052 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188960+0000", 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:25:28.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188966+0000", 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5370000000000003e-06 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188972+0000", 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188977+0000", 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3939999999999998e-06 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209425+0000", 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020447775000000001 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209511+0000", 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6273999999999994e-05 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209612+0000", 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101344 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7062 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571097182999999, 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021205383000000001, 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7062 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189140+0000", 2024-05-08T08:25:28.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189146+0000", 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.716e-06 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189151+0000", 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5820000000000003e-06 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209642+0000", 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020491067000000002 2024-05-08T08:25:28.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209730+0000", 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8078000000000006e-05 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210343+0000", 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061346199999999995 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7063 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 4194304~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:28.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571078639, 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022006687, 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7063 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189157+0000", 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1600000000000002e-07 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189159+0000", 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9570000000000001e-06 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189164+0000", 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189169+0000", 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3680000000000004e-06 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210391+0000", 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021222382000000001 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210478+0000", 2024-05-08T08:25:28.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7168000000000003e-05 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211163+0000", 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000684693 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7064 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571060490000001, 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022198532, 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.793 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7064 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-07 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189177+0000", 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.88e-06 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189182+0000", 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7910000000000001e-06 2024-05-08T08:25:28.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189187+0000", 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211198+0000", 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022011134000000002 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211289+0000", 2024-05-08T08:25:28.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0537000000000004e-05 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211373+0000", 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4426999999999997e-05 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7068 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571042379, 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022446079000000001, 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7068 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:28.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8599999999999998e-07 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189195+0000", 2024-05-08T08:25:28.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189201+0000", 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189216+0000", 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4758e-05 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211430+0000", 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022213944999999999 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211529+0000", 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9504000000000001e-05 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211639+0000", 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010937099999999999 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7069 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.571011399, 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022643852999999999, 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7069 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189227+0000", 2024-05-08T08:25:28.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6410000000000002e-06 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189234+0000", 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189244+0000", 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597e-05 2024-05-08T08:25:28.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211685+0000", 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022441040999999998 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211778+0000", 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2449999999999997e-05 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211867+0000", 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9833999999999994e-05 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7076 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570919947, 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019137119000000001, 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.806 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7076 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189315+0000", 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189317+0000", 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7489999999999999e-06 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189322+0000", 2024-05-08T08:25:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2769999999999996e-06 2024-05-08T08:25:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189327+0000", 2024-05-08T08:25:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2719999999999999e-06 2024-05-08T08:25:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207803+0000", 2024-05-08T08:25:28.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018476037000000001 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207880+0000", 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7377000000000001e-05 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208452+0000", 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00057198700000000004 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.811 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7077 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570901402000001, 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019800081000000001, 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7077 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:28.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189336+0000", 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:25:28.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189342+0000", 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0220000000000001e-06 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189347+0000", 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999998e-06 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208501+0000", 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019154612000000001 2024-05-08T08:25:28.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208603+0000", 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101408 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209134+0000", 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000530938 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7078 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:28.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570882015, 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020694535, 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7078 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189355+0000", 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189361+0000", 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.694e-06 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189366+0000", 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209163+0000", 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01979707 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209236+0000", 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3203999999999999e-05 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210048+0000", 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000811683 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.819 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7079 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 1835008~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570863836000001, 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024715221999999998, 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7079 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:28.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189372+0000", 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189374+0000", 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.052e-06 2024-05-08T08:25:28.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189379+0000", 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8950000000000003e-06 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189384+0000", 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211890+0000", 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022506291000000001 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211938+0000", 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7750999999999997e-05 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214086+0000", 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00214818 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7080 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 4456448~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570844528, 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025413182999999999, 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7080 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189391+0000", 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189394+0000", 2024-05-08T08:25:28.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0230000000000001e-06 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189402+0000", 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189414+0000", 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1639e-05 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214132+0000", 2024-05-08T08:25:28.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024717807000000001 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214218+0000", 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6093000000000004e-05 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214804+0000", 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058589500000000001 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7081 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570802265999999, 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025598254000000001, 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7081 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189436+0000", 2024-05-08T08:25:28.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189443+0000", 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189448+0000", 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0810000000000001e-06 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214836+0000", 2024-05-08T08:25:28.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025387742000000001 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214926+0000", 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0665e-05 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215031+0000", 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104678 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.831 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7085 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:28.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570779128, 2024-05-08T08:25:28.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025822804000000001, 2024-05-08T08:25:28.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.832 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7085 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189459+0000", 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3379999999999999e-06 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189465+0000", 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.088e-06 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189470+0000", 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8640000000000004e-06 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215073+0000", 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025603819999999999 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215180+0000", 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106215 2024-05-08T08:25:28.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215279+0000", 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9087999999999997e-05 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7086 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.570759287, 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026065310000000001, 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:28.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7086 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:28.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189478+0000", 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189486+0000", 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.841e-06 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189491+0000", 2024-05-08T08:25:28.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215315+0000", 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025824427 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215411+0000", 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5606000000000006e-05 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215541+0000", 2024-05-08T08:25:28.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012996899999999999 2024-05-08T08:25:28.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:28.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:28.841 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:28.842 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:28.843 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:28.843 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:28.843 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:28.844 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:28.844 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:28.845 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:28.845 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:28.845 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:28.846 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:28.849+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:28.943 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:28.949+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:28.953 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:28.953 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:28.953 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:28.953 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:28.962 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:29.046 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:29.053+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:29.099 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone11 intact 2024-05-08T08:25:29.115 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:29.115 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:29.116 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:29.116 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:29.116 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:29.124 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:29.145 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:29.152+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:29.206 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-05-08T08:25:29.246 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:29.253+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 22.671023693999999, 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098847239999999992, 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.276 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 9 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:29.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611387+0000", 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.181e-06 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611395+0000", 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0730000000000005e-06 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611411+0000", 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6634999999999999e-05 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611427+0000", 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5699e-05 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611486+0000", 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8692e-05 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611491+0000", 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.989e-06 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615250+0000", 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037591579999999999 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615342+0000", 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2561999999999999e-05 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615480+0000", 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013755399999999999 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.616631+0000", 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011506999999999999 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621178+0000", 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045473620000000001 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621230+0000", 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1845999999999998e-05 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621269+0000", 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9273000000000002e-05 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1683 3.e e23/18 3:777f8546:::rbd_data.119116cc7dd0.0000000000000187:head v 23'35, mlcod=23'33)", 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.161692099, 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059551170000000002, 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.283 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120716+0000", 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2399999999999999e-07 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120725+0000", 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3829999999999992e-06 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120734+0000", 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7099999999999996e-06 2024-05-08T08:25:29.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120739+0000", 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5369999999999997e-06 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120789+0000", 2024-05-08T08:25:29.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0203e-05 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120806+0000", 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7085999999999999e-05 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126597+0000", 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005790936 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126671+0000", 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3837999999999993e-05 2024-05-08T08:25:29.287 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1680 3.1b e23/18 3:dc22c8ef:::rbd_object_map.119116cc7dd0:head v 23'453, mlcod=23'449)", 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.16163002, 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059238520000000003, 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120778+0000", 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120779+0000", 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120784+0000", 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5809999999999996e-06 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120802+0000", 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8063999999999998e-05 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120809+0000", 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6630000000000004e-06 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120863+0000", 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3918000000000001e-05 2024-05-08T08:25:29.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.120878+0000", 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4822e-05 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126657+0000", 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057795650000000004 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.126702+0000", 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4610000000000001e-05 2024-05-08T08:25:29.291 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1826 3.3 3:c3deaffe:::rbd_data.119116cc7dd0.00000000000001e1:head [set-alloc-hint object_size 262144 write_size 262144,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.030984822000001, 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0079090730000000008, 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1826 2024-05-08T08:25:29.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251425+0000", 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.215e-06 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251433+0000", 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9060000000000008e-06 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251444+0000", 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0813e-05 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251449+0000", 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251489+0000", 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9774999999999998e-05 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251542+0000", 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3162999999999999e-05 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:29.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251630+0000", 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7309e-05 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.252128+0000", 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00049835099999999996 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259274+0000", 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071460289999999999 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259308+0000", 2024-05-08T08:25:29.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3679000000000001e-05 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259332+0000", 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4752999999999998e-05 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1976 3.3 3:c4f5f4db:::rbd_data.119116cc7dd0.000000000000003b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.874163368000001, 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065238700000000002, 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.297 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1976 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408245+0000", 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.39e-07 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408247+0000", 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8729999999999999e-06 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408251+0000", 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.357e-06 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408254+0000", 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9699999999999999e-06 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408768+0000", 2024-05-08T08:25:29.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00051321699999999995 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408852+0000", 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4888000000000004e-05 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408963+0000", 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110176 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409769+0000", 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00080602600000000005 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414684+0000", 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049152529999999996 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414731+0000", 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.702e-05 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414769+0000", 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7750999999999998e-05 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1972 3.1d e23/18 3:ba47efeb:::rbd_data.119116cc7dd0.0000000000000037:head v 23'32, mlcod=23'30)", 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.873831794000001, 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0061126660000000001, 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.303 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408576+0000", 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408577+0000", 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3600000000000003e-07 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408583+0000", 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7139999999999998e-06 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408593+0000", 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0429e-05 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408600+0000", 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7209999999999997e-06 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408870+0000", 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00027024599999999997 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408889+0000", 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8961999999999999e-05 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414654+0000", 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057644200000000001 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414689+0000", 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5537999999999998e-05 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1979 3.1b e23/18 3:d8488a8a:::rbd_data.119116cc7dd0.0000000000000086:head v 23'552, mlcod=23'550)", 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.873427651, 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058253389999999997, 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.307 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:29.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.309 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408981+0000", 2024-05-08T08:25:29.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7599999999999997e-07 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408987+0000", 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5149999999999997e-06 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408996+0000", 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.38e-06 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409002+0000", 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1979999999999998e-06 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409049+0000", 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6844000000000001e-05 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.409067+0000", 2024-05-08T08:25:29.312 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7629999999999999e-05 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414772+0000", 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057050579999999998 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414806+0000", 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4137999999999997e-05 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.313 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2374 3.1a 3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.561092651999999, 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011630434, 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.314 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2374 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:29.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1349999999999999e-06 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721322+0000", 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.13e-06 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721340+0000", 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7988000000000001e-05 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721348+0000", 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9789999999999993e-06 2024-05-08T08:25:29.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721408+0000", 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0168999999999999e-05 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721494+0000", 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6001000000000006e-05 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721716+0000", 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022242200000000001 2024-05-08T08:25:29.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.722577+0000", 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00086067299999999995 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732885+0000", 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010308286 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732916+0000", 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0437999999999999e-05 2024-05-08T08:25:29.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732946+0000", 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0213e-05 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7094 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1565839~19667825] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.077196418, 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008628854, 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.319 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7094 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205213+0000", 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.54e-07 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206767+0000", 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001554607 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206779+0000", 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1599e-05 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206785+0000", 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0050000000000001e-06 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206831+0000", 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6108e-05 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206916+0000", 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5575000000000007e-05 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207099+0000", 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000182237 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209859+0000", 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027605020000000002 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213776+0000", 2024-05-08T08:25:29.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003917238 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213810+0000", 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4051999999999999e-05 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213841+0000", 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0176999999999999e-05 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.324 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7095 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.075609579, 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071300790000000001, 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7095 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:25:29.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206801+0000", 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3429999999999998e-06 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206810+0000", 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8389999999999996e-06 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206814+0000", 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1119999999999998e-06 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208857+0000", 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020428310000000002 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208905+0000", 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7849999999999998e-05 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209120+0000", 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000215362 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209862+0000", 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074158700000000004 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213871+0000", 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040092579999999999 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213903+0000", 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1661000000000001e-05 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213929+0000", 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5795e-05 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7096 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.075102565, 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072441390000000001, 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.330 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7096 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207309+0000", 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6960000000000001e-06 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207315+0000", 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207319+0000", 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8840000000000004e-06 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209378+0000", 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020587029999999998 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209422+0000", 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4320000000000003e-05 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209632+0000", 2024-05-08T08:25:29.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020937899999999999 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.210465+0000", 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083359500000000002 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214459+0000", 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039939270000000004 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214514+0000", 2024-05-08T08:25:29.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4338000000000001e-05 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214550+0000", 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6130000000000001e-05 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.011031073, 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.335 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:29.336 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.340 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:7160 3.1a e28/18 ondisk, result = 0)", 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.995498789999999, 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0078836489999999995, 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.341 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286911+0000", 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4580000000000001e-06 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286918+0000", 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9750000000000001e-06 2024-05-08T08:25:29.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286935+0000", 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7337e-05 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286946+0000", 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0611999999999999e-05 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294654+0000", 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.007707958 2024-05-08T08:25:29.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294664+0000", 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0475999999999999e-05 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294793+0000", 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012883299999999999 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:13529 3.14 3:29884244:::rbd_data.119174045df.0000000000000089:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:29.344 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.9936921889999999, 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016452866999999999, 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 13529 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:29.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288717+0000", 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288721+0000", 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9489999999999998e-06 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288730+0000", 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1670000000000005e-06 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288735+0000", 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288772+0000", 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7175000000000002e-05 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288852+0000", 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.291616+0000", 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027647800000000001 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305030+0000", 2024-05-08T08:25:29.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013413625 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305098+0000", 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7991999999999995e-05 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305136+0000", 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8197e-05 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305169+0000", 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2601000000000002e-05 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.349 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14728 3.14 3:2f5e214b:::rbd_data.119174045df.00000000000000c9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.898302771, 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082457929999999995, 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.350 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14728 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384106+0000", 2024-05-08T08:25:29.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6300000000000004e-07 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384110+0000", 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2400000000000001e-06 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384121+0000", 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0190999999999999e-05 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384126+0000", 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384173+0000", 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299000000000002e-05 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384253+0000", 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9802000000000006e-05 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.385620+0000", 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001366843 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392233+0000", 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066128469999999998 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392250+0000", 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7105999999999999e-05 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392301+0000", 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1453999999999999e-05 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.354 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392351+0000", 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9883000000000003e-05 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14789 3.9 3:90dc9cea:::rbd_data.119174045df.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.8259330220000001, 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024649007000000001, 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.355 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14789 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456476+0000", 2024-05-08T08:25:29.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0279999999999999e-06 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456480+0000", 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1280000000000001e-06 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456491+0000", 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0436e-05 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456497+0000", 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.64e-06 2024-05-08T08:25:29.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456545+0000", 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7432999999999998e-05 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456635+0000", 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9859999999999997e-05 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.460437+0000", 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038026510000000002 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481041+0000", 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020603652 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481053+0000", 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2301e-05 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481088+0000", 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4669999999999998e-05 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481124+0000", 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6208000000000003e-05 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:14789 3.9 e37/18 ondisk, result = 0)", 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.8088286120000001, 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074674299999999997, 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473581+0000", 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.232e-06 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473585+0000", 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473599+0000", 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3859000000000001e-05 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473608+0000", 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7509999999999997e-06 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481025+0000", 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074177699999999997 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481037+0000", 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1134e-05 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481047+0000", 2024-05-08T08:25:29.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0353000000000001e-05 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15094 3.9 3:97b56102:::rbd_data.119174045df.0000000000000061:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.4699250699999999, 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019831228999999999, 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.364 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15094 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812484+0000", 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9700000000000005e-07 2024-05-08T08:25:29.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812489+0000", 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4559999999999999e-06 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812502+0000", 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2493999999999999e-05 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812509+0000", 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3919999999999997e-06 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812564+0000", 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4565999999999999e-05 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812693+0000", 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012915299999999999 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.816118+0000", 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003425329 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832226+0000", 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016107360000000001 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832238+0000", 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2500999999999999e-05 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832279+0000", 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0737e-05 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832314+0000", 2024-05-08T08:25:29.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5343999999999999e-05 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:15094 3.9 e37/18 ondisk, result = 0)", 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.826002+0000", 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.456405669, 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006229058, 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826002+0000", 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826002+0000", 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826004+0000", 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.336e-06 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826009+0000", 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3029999999999999e-06 2024-05-08T08:25:29.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826025+0000", 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5549e-05 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.826035+0000", 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8609999999999993e-06 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832212+0000", 2024-05-08T08:25:29.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006177564 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832221+0000", 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7450000000000001e-06 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832232+0000", 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0699999999999999e-05 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15144 3.3 3:c762a68c:::rbd_data.119174045df.0000000000000027:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:29.372 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.3956961269999999, 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099787780000000006, 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15144 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886713+0000", 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4200000000000005e-07 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886717+0000", 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4349999999999999e-06 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886728+0000", 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0535e-05 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886735+0000", 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2139999999999999e-06 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886779+0000", 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4382999999999998e-05 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886827+0000", 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7981e-05 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.888464+0000", 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001636637 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896530+0000", 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0080662830000000005 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896580+0000", 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9484000000000003e-05 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896642+0000", 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896691+0000", 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8848999999999999e-05 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.377 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:29.379 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:29.380 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:29.385+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:29.458 INFO:teuthology.orchestra.run.smithi031.stdout:1965 writesame 0x2160251 thru 0x216b0e4 data_size 0x2(0xae94 bytes) 2024-05-08T08:25:29.480 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:29.486+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:29.549 INFO:teuthology.orchestra.run.smithi031.stdout:1966 read 0x53b04dc thru 0x53bfde6 (0xf90b bytes) 2024-05-08T08:25:29.549 INFO:teuthology.orchestra.run.smithi031.stdout:1967 write 0x25f65fc thru 0x25f793d (0x1342 bytes) 2024-05-08T08:25:29.550 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:29.550 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:29.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.550 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21268 3.1c 3.be8d9cfc (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:29.551 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.558101+0000", 2024-05-08T08:25:29.551 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.6909999999999997e-05, 2024-05-08T08:25:29.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8134000000000003e-05, 2024-05-08T08:25:29.551 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.551 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.551 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:25:29.552 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.552 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.552 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.552 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21268 2024-05-08T08:25:29.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.552 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.558101+0000", 2024-05-08T08:25:29.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.558101+0000", 2024-05-08T08:25:29.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.558102+0000", 2024-05-08T08:25:29.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5990000000000001e-06 2024-05-08T08:25:29.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.558115+0000", 2024-05-08T08:25:29.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2252000000000001e-05 2024-05-08T08:25:29.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.558126+0000", 2024-05-08T08:25:29.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1703e-05 2024-05-08T08:25:29.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.558133+0000", 2024-05-08T08:25:29.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:25:29.557 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.557 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.557 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.558 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.558 INFO:teuthology.orchestra.run.smithi149.stdout: ], 2024-05-08T08:25:29.558 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 1 2024-05-08T08:25:29.558 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:29.559 INFO:teuthology.orchestra.run.smithi031.stdout:1968 read 0x57fc525 thru 0x57fd021 (0xafd bytes) 2024-05-08T08:25:29.560 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:29.563 INFO:teuthology.orchestra.run.smithi031.stdout:1969 write 0xe15c798 thru 0xe15f092 (0x28fb bytes) 2024-05-08T08:25:29.571 INFO:teuthology.orchestra.run.smithi031.stdout:1970 trunc from 0xe15f093 to 0xa61b023 2024-05-08T08:25:29.581 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:29.588+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:29.592 INFO:teuthology.orchestra.run.smithi031.stdout:1971 punch from 0x2885a8b to 0x2887609, (0x1b7e bytes) 2024-05-08T08:25:29.592 INFO:teuthology.orchestra.run.smithi031.stdout:1972 read 0x4ec9685 thru 0x4ed8e3e (0xf7ba bytes) 2024-05-08T08:25:29.593 INFO:teuthology.orchestra.run.smithi031.stdout:1973 read 0xa0b1561 thru 0xa0b56ea (0x418a bytes) 2024-05-08T08:25:29.593 INFO:teuthology.orchestra.run.smithi031.stdout:1974 read 0x360e59f thru 0x361810b (0x9b6d bytes) 2024-05-08T08:25:29.593 INFO:teuthology.orchestra.run.smithi031.stdout:1975 read 0x115295b thru 0x115e0fa (0xb7a0 bytes) 2024-05-08T08:25:29.594 INFO:teuthology.orchestra.run.smithi031.stdout:1976 punch from 0x119bcc1 to 0x11aa7c2, (0xeb01 bytes) 2024-05-08T08:25:29.594 INFO:teuthology.orchestra.run.smithi031.stdout:1977 write 0xdefeb thru 0xe6b78 (0x7b8e bytes) 2024-05-08T08:25:29.595 INFO:teuthology.orchestra.run.smithi031.stdout:1978 read 0x94b0d7f thru 0x94c05d6 (0xf858 bytes) 2024-05-08T08:25:29.596 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:29.596 INFO:teuthology.orchestra.run.smithi031.stdout:1982 punch from 0x4936501 to 0x4939267, (0x2d66 bytes) 2024-05-08T08:25:29.596 INFO:teuthology.orchestra.run.smithi031.stdout:1983 write 0x560d132 thru 0x560ed21 (0x1bf0 bytes) 2024-05-08T08:25:29.610 INFO:teuthology.orchestra.run.smithi031.stdout:1984 writesame 0x1c1ba5e thru 0x1c26e60 data_size 0x1(0xb403 bytes) 2024-05-08T08:25:29.682 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:29.689+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:29.707 INFO:teuthology.orchestra.run.smithi031.stdout:1985 write 0x342d05a thru 0x3438d21 (0xbcc8 bytes) 2024-05-08T08:25:29.709 INFO:teuthology.orchestra.run.smithi031.stdout:1986 trunc from 0xa61b023 to 0xab5c509 2024-05-08T08:25:29.716 INFO:teuthology.orchestra.run.smithi031.stdout:1988 write 0x4bfc6d4 thru 0x4c0b487 (0xedb4 bytes) 2024-05-08T08:25:29.717 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:29.718 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:29.730 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:29.730 INFO:teuthology.orchestra.run.smithi031.stdout:1990 read 0x78fb11e thru 0x790af77 (0xfe5a bytes) 2024-05-08T08:25:29.730 INFO:teuthology.orchestra.run.smithi031.stdout:1991 read 0x6eb10c5 thru 0x6eb1aa5 (0x9e1 bytes) 2024-05-08T08:25:29.731 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:29.731 INFO:teuthology.orchestra.run.smithi031.stdout:1993 read 0x725d762 thru 0x7268cdb (0xb57a bytes) 2024-05-08T08:25:29.731 INFO:teuthology.orchestra.run.smithi031.stdout:1994 write 0x3388c1a thru 0x338c8f6 (0x3cdd bytes) 2024-05-08T08:25:29.731 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:29.732 INFO:teuthology.orchestra.run.smithi031.stdout:1995 punch from 0x6b77637 to 0x6b86d9b, (0xf764 bytes) 2024-05-08T08:25:29.740 INFO:teuthology.orchestra.run.smithi031.stdout:1998 write 0xcd5bcd4 thru 0xcd611b9 (0x54e6 bytes) 2024-05-08T08:25:29.743 INFO:teuthology.orchestra.run.smithi031.stdout:2000 read 0x192e37 thru 0x19e800 (0xb9ca bytes) 2024-05-08T08:25:29.745 INFO:teuthology.orchestra.run.smithi031.stdout:2004 punch from 0x598ba24 to 0x5993d0e, (0x82ea bytes) 2024-05-08T08:25:29.745 INFO:teuthology.orchestra.run.smithi031.stdout:2005 write 0xc1d71ba thru 0xc1de2ab (0x70f2 bytes) 2024-05-08T08:25:29.757 INFO:teuthology.orchestra.run.smithi031.stdout:2006 write 0xe2079f5 thru 0xe20d3ed (0x59f9 bytes) 2024-05-08T08:25:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:2008 punch from 0x877f0c2 to 0x878751e, (0x845c bytes) 2024-05-08T08:25:29.761 INFO:teuthology.orchestra.run.smithi031.stdout:2009 writesame 0x81ccd55 thru 0x81d8afb data_size 0x1(0xbda7 bytes) 2024-05-08T08:25:29.777 INFO:teuthology.orchestra.run.smithi031.stdout:2010 read 0x6df1c3f thru 0x6dff85d (0xdc1f bytes) 2024-05-08T08:25:29.777 INFO:teuthology.orchestra.run.smithi031.stdout:2011 read 0x94f9cdf thru 0x950238a (0x86ac bytes) 2024-05-08T08:25:29.778 INFO:teuthology.orchestra.run.smithi031.stdout:2012 write 0x54cc20a thru 0x54d5fa5 (0x9d9c bytes) 2024-05-08T08:25:29.779 INFO:teuthology.orchestra.run.smithi031.stdout:2014 write 0x91665dc thru 0x916e18a (0x7baf bytes) 2024-05-08T08:25:29.783 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:29.790+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:29.791 INFO:teuthology.orchestra.run.smithi031.stdout:2015 trunc from 0xe20d3ee to 0x5f70771 2024-05-08T08:25:29.804 INFO:teuthology.orchestra.run.smithi031.stdout:2016 write 0x5732456 thru 0x57376be (0x5269 bytes) 2024-05-08T08:25:29.806 INFO:teuthology.orchestra.run.smithi031.stdout:2017 read 0xd9db67 thru 0xda7556 (0x99f0 bytes) 2024-05-08T08:25:29.806 INFO:teuthology.orchestra.run.smithi031.stdout:2018 punch from 0x2d2fc3f to 0x2d3ee7f, (0xf240 bytes) 2024-05-08T08:25:29.806 INFO:teuthology.orchestra.run.smithi031.stdout:2019 writesame 0x5edeeb thru 0x5f5201 data_size 0x45(0x7317 bytes) 2024-05-08T08:25:29.808 INFO:teuthology.orchestra.run.smithi031.stdout:2020 writesame 0x3fa2e0f thru 0x3fafe96 data_size 0x4(0xd088 bytes) 2024-05-08T08:25:29.814 INFO:teuthology.orchestra.run.smithi031.stdout:2022 write 0x39a03b0 thru 0x39a1d70 (0x19c1 bytes) 2024-05-08T08:25:29.816 INFO:teuthology.orchestra.run.smithi031.stdout:2023 write 0x93612b thru 0x93ad31 (0x4c07 bytes) 2024-05-08T08:25:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:2025 punch from 0x57e29b to 0x58a0cf, (0xbe34 bytes) 2024-05-08T08:25:29.818 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:29.819 INFO:teuthology.orchestra.run.smithi031.stdout:2027 trunc from 0x5f70771 to 0x50fe160 2024-05-08T08:25:29.835 INFO:teuthology.orchestra.run.smithi031.stdout:2028 trunc from 0x50fe160 to 0xc20601b 2024-05-08T08:25:29.841 INFO:teuthology.orchestra.run.smithi031.stdout:2029 write 0x3013714 thru 0x301fb46 (0xc433 bytes) 2024-05-08T08:25:29.843 INFO:teuthology.orchestra.run.smithi031.stdout:2030 read 0xab42fb1 thru 0xab452f8 (0x2348 bytes) 2024-05-08T08:25:29.844 INFO:teuthology.orchestra.run.smithi031.stdout:2031 write 0x8480b13 thru 0x8486078 (0x5566 bytes) 2024-05-08T08:25:29.845 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:29.845 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1700 3.1d 3:bbece77f:::rbd_data.119116cc7dd0.0000000000000263:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.712208371999999, 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0049688420000000002, 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.846 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1700 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139607+0000", 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3900000000000004e-07 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139611+0000", 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7419999999999999e-06 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139621+0000", 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6369999999999994e-06 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139636+0000", 2024-05-08T08:25:29.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5367e-05 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139684+0000", 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899000000000003e-05 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139733+0000", 2024-05-08T08:25:29.849 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9205000000000001e-05 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.139798+0000", 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4431999999999996e-05 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144321+0000", 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045238159999999999 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.850 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144511+0000", 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018977100000000001 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144544+0000", 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2620000000000003e-05 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.144575+0000", 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1714000000000002e-05 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.851 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1972 3.1d 3:ba47efeb:::rbd_data.119116cc7dd0.0000000000000037:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.443566053000001, 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066697570000000001, 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1972 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.852 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408249+0000", 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408250+0000", 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.026e-06 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408256+0000", 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8490000000000002e-06 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408268+0000", 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1827999999999999e-05 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408275+0000", 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:29.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408331+0000", 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5989000000000002e-05 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408406+0000", 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5666000000000004e-05 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.408509+0000", 2024-05-08T08:25:29.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010266400000000001 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.410688+0000", 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002179112 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414827+0000", 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041387699999999999 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414878+0000", 2024-05-08T08:25:29.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1480999999999997e-05 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414919+0000", 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0188999999999998e-05 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1989 3.1c 3:39f766e8:::rbd_data.119116cc7dd0.0000000000000145:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.439439046, 2024-05-08T08:25:29.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005150277, 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1989 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412376+0000", 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412377+0000", 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3600000000000003e-07 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412380+0000", 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7639999999999999e-06 2024-05-08T08:25:29.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412390+0000", 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9729999999999992e-06 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412396+0000", 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2519999999999999e-06 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412447+0000", 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1177999999999997e-05 2024-05-08T08:25:29.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412525+0000", 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8375999999999996e-05 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412612+0000", 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7106000000000003e-05 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415079+0000", 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024662569999999999 2024-05-08T08:25:29.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417466+0000", 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023874170000000002 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417498+0000", 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2215000000000002e-05 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417526+0000", 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8002999999999999e-05 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1994 3.5 3:a2dbaef4:::rbd_data.119116cc7dd0.0000000000000174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.436582977, 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052774390000000001, 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.863 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1994 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415232+0000", 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415233+0000", 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.61e-07 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415238+0000", 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7309999999999998e-06 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415250+0000", 2024-05-08T08:25:29.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.118e-05 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415256+0000", 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.122e-06 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415308+0000", 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2111000000000003e-05 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415389+0000", 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0889000000000004e-05 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415492+0000", 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000103316 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416315+0000", 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000822804 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.867 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420358+0000", 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004043128 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420470+0000", 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000112197 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420509+0000", 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9199999999999997e-05 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.868 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1995 3.17 e23/18 3:eb8d97a8:::rbd_data.119116cc7dd0.0000000000000175:head v 23'20, mlcod=23'18)", 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.436079122999999, 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072714820000000001, 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:29.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415736+0000", 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9200000000000001e-07 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.870 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415742+0000", 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415751+0000", 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2890000000000003e-06 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415757+0000", 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0190000000000003e-06 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.871 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415804+0000", 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7141999999999997e-05 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415824+0000", 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0136000000000002e-05 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.422967+0000", 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071422839999999996 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423007+0000", 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0720000000000003e-05 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2002 3.1f e23/18 3:fbb43922:::rbd_data.119116cc7dd0.00000000000001e2:head v 23'25, mlcod=23'23)", 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.433923843999999, 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051483420000000002, 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.873 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417891+0000", 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417892+0000", 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417896+0000", 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9339999999999999e-06 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417905+0000", 2024-05-08T08:25:29.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8379999999999997e-06 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417914+0000", 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9509999999999995e-06 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417969+0000", 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4225999999999999e-05 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417988+0000", 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9318999999999999e-05 2024-05-08T08:25:29.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423010+0000", 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050221659999999998 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.423039+0000", 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9417999999999999e-05 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2011 3.16 3:6eef803b:::rbd_data.119116cc7dd0.0000000000000257:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.430863456000001, 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050707879999999997, 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2011 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420952+0000", 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1299999999999998e-07 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420957+0000", 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170000000000001e-06 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420969+0000", 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1938e-05 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420976+0000", 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3200000000000002e-06 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421035+0000", 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8696999999999997e-05 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421080+0000", 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4709000000000002e-05 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.421139+0000", 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9762999999999997e-05 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425663+0000", 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045239520000000004 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425971+0000", 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00030778899999999999 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.425999+0000", 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8005e-05 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.426022+0000", 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3184999999999999e-05 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2126 3.16 3:6a294600:::rbd_data.119116cc7dd0.000000000000015b:head [set-alloc-hint object_size 262144 write_size 262144,write 59392~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.330917516, 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066558700000000004, 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.883 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2126 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520897+0000", 2024-05-08T08:25:29.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520898+0000", 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7999999999999993e-07 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520908+0000", 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2520000000000005e-06 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520920+0000", 2024-05-08T08:25:29.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1891e-05 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520927+0000", 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4950000000000002e-06 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.520980+0000", 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2895000000000001e-05 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521065+0000", 2024-05-08T08:25:29.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4698999999999997e-05 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521193+0000", 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012812199999999999 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.521826+0000", 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063310000000000005 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527495+0000", 2024-05-08T08:25:29.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056689669999999996 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527529+0000", 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4394000000000003e-05 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.527553+0000", 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4074999999999999e-05 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.888 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2133 3.16 3:6a294600:::rbd_data.119116cc7dd0.000000000000015b:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.311604346999999, 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065993600000000003, 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2133 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:29.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540211+0000", 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7800000000000001e-07 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540215+0000", 2024-05-08T08:25:29.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7689999999999998e-06 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540225+0000", 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0039e-05 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540230+0000", 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2830000000000001e-06 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540280+0000", 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9480000000000001e-05 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540352+0000", 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2347000000000004e-05 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.540409+0000", 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6518999999999999e-05 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541006+0000", 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00059697199999999998 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546717+0000", 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057116579999999997 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546764+0000", 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7039000000000001e-05 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546810+0000", 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5575999999999999e-05 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2140 3.5 3:a1b38341:::rbd_data.119116cc7dd0.000000000000027c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.309962984999999, 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00497986, 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.894 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2140 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541852+0000", 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541853+0000", 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9699999999999995e-07 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541857+0000", 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5469999999999998e-06 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541868+0000", 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0505e-05 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541873+0000", 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0989999999999998e-06 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.541923+0000", 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0383000000000002e-05 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542000+0000", 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6960999999999997e-05 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542106+0000", 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000105717 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.542543+0000", 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00043726399999999998 2024-05-08T08:25:29.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042138540000000004 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546793+0000", 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6269000000000001e-05 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546832+0000", 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8464000000000003e-05 2024-05-08T08:25:29.899 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2145 3.1d 3:bdb88d54:::rbd_data.119116cc7dd0.0000000000000326:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.305058446, 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006666384, 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2145 2024-05-08T08:25:29.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546757+0000", 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2e-07 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546760+0000", 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8990000000000001e-06 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546766+0000", 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5390000000000004e-06 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.546773+0000", 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.19e-06 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547312+0000", 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053938900000000004 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547380+0000", 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8189999999999996e-05 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.547474+0000", 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3450999999999997e-05 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.550283+0000", 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002809228 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553330+0000", 2024-05-08T08:25:29.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030471249999999999 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553378+0000", 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7710000000000002e-05 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.553423+0000", 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5142999999999998e-05 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2197 3.1d 3:be511a0a:::rbd_data.119116cc7dd0.0000000000000209:head [set-alloc-hint object_size 262144 write_size 262144,write 4096~6144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.261704451, 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004962401, 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2197 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590111+0000", 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590112+0000", 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0419999999999999e-06 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590120+0000", 2024-05-08T08:25:29.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5050000000000007e-06 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590131+0000", 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.077e-05 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590136+0000", 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590180+0000", 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3299e-05 2024-05-08T08:25:29.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590259+0000", 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8848000000000001e-05 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.590343+0000", 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4328999999999998e-05 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.592143+0000", 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001800248 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595017+0000", 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002873553 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595049+0000", 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2481999999999998e-05 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.595073+0000", 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.37e-05 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.910 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2338 3.16 3:6f361ecb:::rbd_data.119116cc7dd0.000000000000001b:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.144297656999999, 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052194750000000003, 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.911 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2338 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707517+0000", 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707518+0000", 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1620000000000001e-06 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707525+0000", 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7900000000000002e-06 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707549+0000", 2024-05-08T08:25:29.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3414999999999999e-05 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707558+0000", 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1600000000000004e-06 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707639+0000", 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0772000000000005e-05 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707722+0000", 2024-05-08T08:25:29.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3366000000000001e-05 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707911+0000", 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000189447 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.709463+0000", 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001551621 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712639+0000", 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031764419999999998 2024-05-08T08:25:29.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712693+0000", 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3402000000000001e-05 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712737+0000", 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3897999999999999e-05 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2713 3.2 e24/18 3:42c14b71:::rbd_data.1191a2472683.0000000000000000:head v 24'16, mlcod=23'14)", 2024-05-08T08:25:29.916 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.818527817, 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058213900000000001, 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033287+0000", 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.033289+0000", 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036196+0000", 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0029067730000000001 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036213+0000", 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7394e-05 2024-05-08T08:25:29.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036228+0000", 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4841000000000001e-05 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036289+0000", 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1091000000000002e-05 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.036310+0000", 2024-05-08T08:25:29.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0951e-05 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.039072+0000", 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002761763 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.039109+0000", 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6656999999999998e-05 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2782 3.1c 3:3afc57c8:::rbd_data.1191a2472683.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 23855104~19711] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.79508641, 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010424643000000001, 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2782 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056729+0000", 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.056730+0000", 2024-05-08T08:25:29.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9699999999999994e-07 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061776+0000", 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050462679999999996 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061789+0000", 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3358e-05 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061795+0000", 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1349999999999997e-06 2024-05-08T08:25:29.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061844+0000", 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8414000000000001e-05 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.061957+0000", 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000112759 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.062210+0000", 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000253918 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.066162+0000", 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039516179999999996 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067075+0000", 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00091328600000000002 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067116+0000", 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0599999999999998e-05 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:12.067153+0000", 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7289999999999997e-05 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14729 3.1c 3:3a8d2cd7:::rbd_data.119174045df.00000000000000ca:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.384288+0000", 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.4675273720000002, 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018281189, 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.926 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14729 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384288+0000", 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384288+0000", 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384289+0000", 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1699999999999997e-07 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384294+0000", 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3199999999999999e-06 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384306+0000", 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2296999999999999e-05 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384313+0000", 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1910000000000003e-06 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384359+0000", 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5759e-05 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384448+0000", 2024-05-08T08:25:29.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8547000000000005e-05 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.387555+0000", 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003107014 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402424+0000", 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014869413999999999 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402495+0000", 2024-05-08T08:25:29.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0641000000000003e-05 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402534+0000", 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9387000000000001e-05 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402569+0000", 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4702000000000001e-05 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:14729 3.1c e37/18 ondisk, result = 0)", 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.4545514490000002, 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053112380000000002, 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397264+0000", 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7000000000000003e-07 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397268+0000", 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.743e-06 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397278+0000", 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0231000000000001e-05 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.397284+0000", 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7459999999999997e-06 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402483+0000", 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051991859999999997 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402491+0000", 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5229999999999998e-06 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.402575+0000", 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3938999999999997e-05 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21267 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:29.935 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.33878814200000001, 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044009312000000002, 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21267 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513028+0000", 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.598e-06 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525553+0000", 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.012524067999999999 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525577+0000", 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4771999999999999e-05 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525605+0000", 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7577e-05 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525700+0000", 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5251e-05 2024-05-08T08:25:29.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525885+0000", 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018491199999999999 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.530129+0000", 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042443949999999998 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556920+0000", 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02679107 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556958+0000", 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7644000000000001e-05 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556998+0000", 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0216999999999998e-05 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.557036+0000", 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7808000000000001e-05 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.940 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:21267 3.1c e40/18 ondisk, result = 0)", 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.548031+0000", 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.30378416499999999, 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0090137979999999999, 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.548031+0000", 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.548031+0000", 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.548032+0000", 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1480000000000001e-06 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.548039+0000", 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:25:29.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.548052+0000", 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3251000000000001e-05 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.548060+0000", 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0320000000000003e-06 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556946+0000", 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088862420000000008 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556954+0000", 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1669999999999999e-06 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.557045+0000", 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0409999999999997e-05 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:29.944 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.19098716199999999, 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:29.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:25:29.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.949 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:29.950 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:29.950 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:29.952 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:29.953 INFO:teuthology.orchestra.run.smithi031.stdout:2032 read 0xb3ced53 thru 0xb3db892 (0xcb40 bytes) 2024-05-08T08:25:29.953 INFO:teuthology.orchestra.run.smithi031.stdout:2033 trunc from 0xc20601b to 0x6d362f2 2024-05-08T08:25:29.953 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:29.953 INFO:teuthology.orchestra.run.smithi031.stdout:2035 writesame 0x2698046 thru 0x26a6916 data_size 0x1(0xe8d1 bytes) 2024-05-08T08:25:29.953 INFO:teuthology.orchestra.run.smithi031.stdout:2036 flatten 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2039 read 0x2e2e1cb thru 0x2e34e55 (0x6c8b bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2040 read 0x409da33 thru 0x40aa6a3 (0xcc71 bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2041 punch from 0x13da76 to 0x14ba07, (0xdf91 bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2042 writesame 0x6fd0e4a thru 0x6fd6078 data_size 0x1(0x522f bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2043 read 0x5c24bad thru 0x5c33ef0 (0xf344 bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2044 trunc from 0x6fd6079 to 0x7366803 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2045 write 0x245e539 thru 0x246c936 (0xe3fe bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2047 write 0x2d85661 thru 0x2d85b2a (0x4ca bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2048 read 0x6583f88 thru 0x658e231 (0xa2aa bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2049 read 0x60ccbaf thru 0x60db91d (0xed6f bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2051 write 0xcc77aaf thru 0xcc819ed (0x9f3f bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2052 writesame 0x353aa8f thru 0x35498d1 data_size 0xb(0xee43 bytes) 2024-05-08T08:25:29.954 INFO:teuthology.orchestra.run.smithi031.stdout:2053 trunc from 0xcc819ee to 0x84bd02a 2024-05-08T08:25:29.955 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:29.958+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:29.959 INFO:teuthology.orchestra.run.smithi031.stdout:2054 write 0x47ec555 thru 0x47f33b1 (0x6e5d bytes) 2024-05-08T08:25:29.968 INFO:teuthology.orchestra.run.smithi031.stdout:2056 writesame 0x935dbea thru 0x93685a7 data_size 0x1(0xa9be bytes) 2024-05-08T08:25:29.969 INFO:teuthology.orchestra.run.smithi031.stdout:2057 clone 13 order 24 su 1048576 sc 8 2024-05-08T08:25:30.052 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:30.059+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:30.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:25:30.063 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:30.063 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:30.063 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:30.063 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:30.073 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:30.153 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:30.160+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:30.180 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:30.181 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:30.181 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:30.181 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:30.181 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:30.190 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:30.212 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-05-08T08:25:30.255 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:30.262+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:30.344 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1739 3.a 3:516cf025:::rbd_data.119116cc7dd0.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 90112~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.178958+0000", 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 20.171609917000001, 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039215049999999996, 2024-05-08T08:25:30.345 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.346 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.346 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.346 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.346 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.346 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.346 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1739 2024-05-08T08:25:30.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.347 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178958+0000", 2024-05-08T08:25:30.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178958+0000", 2024-05-08T08:25:30.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.349 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.349 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178960+0000", 2024-05-08T08:25:30.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4589999999999999e-06 2024-05-08T08:25:30.349 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.350 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.350 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178973+0000", 2024-05-08T08:25:30.350 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.364e-05 2024-05-08T08:25:30.350 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.350 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.350 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178986+0000", 2024-05-08T08:25:30.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2678e-05 2024-05-08T08:25:30.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178993+0000", 2024-05-08T08:25:30.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2269999999999996e-06 2024-05-08T08:25:30.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179041+0000", 2024-05-08T08:25:30.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7710999999999998e-05 2024-05-08T08:25:30.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179138+0000", 2024-05-08T08:25:30.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6791000000000002e-05 2024-05-08T08:25:30.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.354 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.354 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179260+0000", 2024-05-08T08:25:30.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000122087 2024-05-08T08:25:30.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.355 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179955+0000", 2024-05-08T08:25:30.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00069502699999999997 2024-05-08T08:25:30.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.182776+0000", 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028215900000000001 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.182838+0000", 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1700999999999998e-05 2024-05-08T08:25:30.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.182880+0000", 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1594000000000002e-05 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.358 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1815 3.10 3:09da06bb:::rbd_data.119116cc7dd0.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:30.358 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:30.358 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 20.109368225000001, 2024-05-08T08:25:30.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057340569999999999, 2024-05-08T08:25:30.358 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.358 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1815 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:30.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241202+0000", 2024-05-08T08:25:30.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:25:30.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241211+0000", 2024-05-08T08:25:30.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6940000000000001e-06 2024-05-08T08:25:30.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241228+0000", 2024-05-08T08:25:30.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7487e-05 2024-05-08T08:25:30.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241239+0000", 2024-05-08T08:25:30.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0954999999999999e-05 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241305+0000", 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5696000000000003e-05 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241432+0000", 2024-05-08T08:25:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000127639 2024-05-08T08:25:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.242178+0000", 2024-05-08T08:25:30.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074582499999999998 2024-05-08T08:25:30.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.245608+0000", 2024-05-08T08:25:30.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003430191 2024-05-08T08:25:30.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246869+0000", 2024-05-08T08:25:30.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001261096 2024-05-08T08:25:30.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246904+0000", 2024-05-08T08:25:30.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5024999999999998e-05 2024-05-08T08:25:30.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246934+0000", 2024-05-08T08:25:30.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9519999999999999e-05 2024-05-08T08:25:30.369 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.369 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.369 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.369 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1817 3.a 3:525853b5:::rbd_data.119116cc7dd0.0000000000000148:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:30.369 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.244093+0000", 2024-05-08T08:25:30.370 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 20.106475231000001, 2024-05-08T08:25:30.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003967043, 2024-05-08T08:25:30.370 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.370 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.370 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.370 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.371 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.371 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.371 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1817 2024-05-08T08:25:30.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.371 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244093+0000", 2024-05-08T08:25:30.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244093+0000", 2024-05-08T08:25:30.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244094+0000", 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0559999999999999e-06 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244099+0000", 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8969999999999999e-06 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244109+0000", 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0394000000000001e-05 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244114+0000", 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2009999999999998e-06 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244167+0000", 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2867000000000001e-05 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244305+0000", 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000137494 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244964+0000", 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000659601 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.247758+0000", 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027938189999999999 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.248002+0000", 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000243584 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.248031+0000", 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8906000000000001e-05 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.248060+0000", 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9224e-05 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1904 3.10 3:0d6c878d:::rbd_data.119116cc7dd0.0000000000000377:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.328225+0000", 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 20.022342881, 2024-05-08T08:25:30.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025701019999999999, 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1904 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328225+0000", 2024-05-08T08:25:30.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328225+0000", 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328227+0000", 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3179999999999999e-06 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328236+0000", 2024-05-08T08:25:30.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9059999999999997e-06 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328248+0000", 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1883e-05 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328254+0000", 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8830000000000003e-06 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328308+0000", 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3431999999999999e-05 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328398+0000", 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0149999999999996e-05 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328481+0000", 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2947000000000004e-05 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.329965+0000", 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001483881 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330734+0000", 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00076957600000000001 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330766+0000", 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2206000000000003e-05 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330795+0000", 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8920000000000001e-05 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1944 3.10 3:0d6c878d:::rbd_data.119116cc7dd0.0000000000000377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.371574+0000", 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.978993879000001, 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037156670000000002, 2024-05-08T08:25:30.383 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.384 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.384 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.384 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.384 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.384 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.384 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1944 2024-05-08T08:25:30.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.385 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371574+0000", 2024-05-08T08:25:30.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371574+0000", 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371576+0000", 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4750000000000001e-06 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371582+0000", 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371599+0000", 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6833999999999998e-05 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371607+0000", 2024-05-08T08:25:30.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2369999999999992e-06 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371672+0000", 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4795999999999994e-05 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371764+0000", 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1527999999999995e-05 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:30.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371865+0000", 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000101094 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.372589+0000", 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00072437300000000005 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.375181+0000", 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025916009999999998 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.375243+0000", 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1867999999999998e-05 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.375290+0000", 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7250000000000003e-05 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1990 3.10 3:09da06bb:::rbd_data.119116cc7dd0.0000000000000146:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.412463+0000", 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.938105622999998, 2024-05-08T08:25:30.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025719789999999998, 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1990 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412463+0000", 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412463+0000", 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412464+0000", 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4100000000000001e-06 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412469+0000", 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:25:30.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412481+0000", 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1951e-05 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412486+0000", 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412538+0000", 2024-05-08T08:25:30.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1539000000000002e-05 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412593+0000", 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5114999999999997e-05 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412667+0000", 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4078000000000005e-05 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.413251+0000", 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058439400000000002 2024-05-08T08:25:30.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414941+0000", 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016893259999999999 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414988+0000", 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7324000000000003e-05 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415034+0000", 2024-05-08T08:25:30.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6554000000000002e-05 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1992 3.a 3:525853b5:::rbd_data.119116cc7dd0.0000000000000148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.935629596999998, 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005824053, 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1992 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414940+0000", 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.029e-06 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414946+0000", 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2890000000000003e-06 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414957+0000", 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1280999999999999e-05 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414966+0000", 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415026+0000", 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0624000000000001e-05 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415093+0000", 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6752000000000001e-05 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415154+0000", 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1016e-05 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417565+0000", 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024105730000000001 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420666+0000", 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031012219999999998 2024-05-08T08:25:30.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420721+0000", 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5260000000000003e-05 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420763+0000", 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1519e-05 2024-05-08T08:25:30.402 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1999 3.1e e23/18 3:7e7612a5:::rbd_data.119116cc7dd0.0000000000000179:head v 23'16, mlcod=23'14)", 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.416681+0000", 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.933886933, 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040606110000000004, 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416681+0000", 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416681+0000", 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416682+0000", 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1299999999999999e-07 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416687+0000", 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9969999999999998e-06 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416696+0000", 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0620000000000007e-06 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416702+0000", 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416742+0000", 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0893000000000003e-05 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416757+0000", 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4686e-05 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420701+0000", 2024-05-08T08:25:30.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039436419999999998 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420742+0000", 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1050999999999997e-05 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2312 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.692314+0000", 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.658254206999999, 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002844868, 2024-05-08T08:25:30.407 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2312 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692314+0000", 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692314+0000", 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692316+0000", 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5600000000000001e-06 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692329+0000", 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2933e-05 2024-05-08T08:25:30.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692363+0000", 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3213000000000002e-05 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692378+0000", 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5753e-05 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692510+0000", 2024-05-08T08:25:30.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013140100000000001 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692521+0000", 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1143e-05 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694159+0000", 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016379910000000001 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694222+0000", 2024-05-08T08:25:30.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2932e-05 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694337+0000", 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00011511899999999999 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694869+0000", 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053169100000000004 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.695100+0000", 2024-05-08T08:25:30.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000231177 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.695133+0000", 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3526999999999999e-05 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.695159+0000", 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5428000000000001e-05 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2330 3.a 3:516cf025:::rbd_data.119116cc7dd0.0000000000000013:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.707370+0000", 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.643198605999999, 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042311270000000003, 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2330 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707370+0000", 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707370+0000", 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707371+0000", 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0589999999999999e-06 2024-05-08T08:25:30.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707376+0000", 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-06 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707392+0000", 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6047000000000001e-05 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707400+0000", 2024-05-08T08:25:30.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.106e-06 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707476+0000", 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6145999999999999e-05 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707556+0000", 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9507999999999998e-05 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707750+0000", 2024-05-08T08:25:30.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019481899999999999 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708939+0000", 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001188789 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711503+0000", 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002563835 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711558+0000", 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4619e-05 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711601+0000", 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2959e-05 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2334 3.1e e24/18 3:7c0c4f5a:::rbd_data.119116cc7dd0.0000000000000017:head v 24'23, mlcod=23'20)", 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.707980+0000", 2024-05-08T08:25:30.419 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.642587810999999, 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040302740000000004, 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707980+0000", 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707980+0000", 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707981+0000", 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707990+0000", 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0289999999999994e-06 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708029+0000", 2024-05-08T08:25:30.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9496999999999998e-05 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708037+0000", 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1130000000000001e-06 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708122+0000", 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4745000000000003e-05 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708151+0000", 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8765e-05 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711963+0000", 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038124299999999999 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712011+0000", 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299999999999998e-05 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:6431 3.10 3:0ebe7e22:::rbd_object_map.119152d8614e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:14.750478+0000", 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.600090547000001, 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024533799999999998, 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 6431 2024-05-08T08:25:30.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750478+0000", 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750478+0000", 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750478+0000", 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.92e-07 2024-05-08T08:25:30.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750482+0000", 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8709999999999999e-06 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750492+0000", 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5070000000000006e-06 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750498+0000", 2024-05-08T08:25:30.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3980000000000004e-06 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750539+0000", 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0694e-05 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750587+0000", 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8495000000000002e-05 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.750788+0000", 2024-05-08T08:25:30.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020095599999999999 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.751418+0000", 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00062998399999999999 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.752878+0000", 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001460102 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.752909+0000", 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0335999999999999e-05 2024-05-08T08:25:30.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:14.752931+0000", 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2345e-05 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:6773 3.1e e28/18 3:7afff813:::rbd_data.119152d8614e.0000000000000002:head v 28'51, mlcod=27'48)", 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287407246000001, 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0032001339999999999, 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.429 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1199999999999999e-07 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063165+0000", 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0210000000000001e-06 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063175+0000", 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9059999999999997e-06 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063182+0000", 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7769999999999997e-06 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063223+0000", 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0883999999999997e-05 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063235+0000", 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2764e-05 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.066316+0000", 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030807690000000001 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.066361+0000", 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4801e-05 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:6777 3.10 3:0ebe7e22:::rbd_object_map.119152d8614e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:30.433 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.071541+0000", 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.279026645, 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0026104549999999998, 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 6777 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071541+0000", 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071541+0000", 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071543+0000", 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1039999999999999e-06 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071548+0000", 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2109999999999997e-06 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071559+0000", 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.168e-05 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071566+0000", 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7530000000000004e-06 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071611+0000", 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4520000000000001e-05 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071683+0000", 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1950000000000001e-05 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071905+0000", 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0002218 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.072542+0000", 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063779799999999997 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.074072+0000", 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001529519 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.074106+0000", 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4461000000000001e-05 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.074152+0000", 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5658999999999998e-05 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7101 3.a 3:570608b4:::rbd_data.119120fdbc10.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1310720~19922944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.135499132, 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053734150000000003, 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.439 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7101 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215071+0000", 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4210000000000002e-06 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217644+0000", 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025721239999999999 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217664+0000", 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0492999999999998e-05 2024-05-08T08:25:30.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217681+0000", 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6985000000000001e-05 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217759+0000", 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7749000000000004e-05 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217900+0000", 2024-05-08T08:25:30.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141361 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.218122+0000", 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022196499999999999 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220140+0000", 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020175129999999999 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220373+0000", 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023305999999999999 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220410+0000", 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7146e-05 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220442+0000", 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2598000000000002e-05 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14271 3.a 3:56b37913:::rbd_data.119174045df.000000000000002e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:24.891244+0000", 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4593244299999997, 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040489970000000004, 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14271 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.445 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891244+0000", 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891244+0000", 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891245+0000", 2024-05-08T08:25:30.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4649999999999999e-06 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891268+0000", 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2886000000000001e-05 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891283+0000", 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4848000000000001e-05 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891307+0000", 2024-05-08T08:25:30.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4031000000000001e-05 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891356+0000", 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8696000000000003e-05 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891452+0000", 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6034999999999998e-05 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.892116+0000", 2024-05-08T08:25:30.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00066427399999999996 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.894982+0000", 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028657869999999998 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.895233+0000", 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025091499999999999 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.895264+0000", 2024-05-08T08:25:30.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1282999999999998e-05 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.895293+0000", 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8776999999999999e-05 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21104 3.a 3:522bcd46:::rbd_data.1191948f0501.000000000000002f:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.476053+0000", 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.874514652, 2024-05-08T08:25:30.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030756070000000002, 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21104 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476053+0000", 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476053+0000", 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476054+0000", 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.82e-07 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476057+0000", 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7480000000000002e-06 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476067+0000", 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0679000000000001e-05 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476073+0000", 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478957+0000", 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028839999999999998 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.479043+0000", 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5500000000000005e-05 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.479129+0000", 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6300999999999999e-05 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.454 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21166 3.10 3:0c68b0bf:::rbd_data.1191948f0501.0000000000000069:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.476221+0000", 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.87434691499999995, 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024855939999999998, 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21166 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476221+0000", 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476221+0000", 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476222+0000", 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476224+0000", 2024-05-08T08:25:30.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5600000000000001e-06 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476231+0000", 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6560000000000003e-06 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476236+0000", 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478567+0000", 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023310869999999999 2024-05-08T08:25:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478643+0000", 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5512000000000005e-05 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478707+0000", 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4185999999999998e-05 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.458 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21167 3.10 3:0c68b0bf:::rbd_data.1191948f0501.0000000000000069:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.87432319700000005, 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0026289339999999999, 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21167 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476250+0000", 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1980000000000003e-06 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476258+0000", 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1620000000000008e-06 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476263+0000", 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4759999999999997e-06 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478732+0000", 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024690300000000001 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478799+0000", 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6427000000000006e-05 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478874+0000", 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5189000000000001e-05 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.462 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:22230 3.a 3:545786ae:::rbd_id.image_client.0-clone13:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.082473+0000", 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.26809506900000002, 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002459078, 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 22230 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.463 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082473+0000", 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082473+0000", 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082474+0000", 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.407e-06 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082481+0000", 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4069999999999998e-06 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082493+0000", 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2598e-05 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082504+0000", 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0659000000000001e-05 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082565+0000", 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1308000000000003e-05 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082573+0000", 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5769999999999996e-06 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.083879+0000", 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013064109999999999 2024-05-08T08:25:30.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.083955+0000", 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5315999999999995e-05 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084059+0000", 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010422 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084714+0000", 2024-05-08T08:25:30.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00065502900000000001 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084878+0000", 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000163695 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084908+0000", 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0361e-05 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084932+0000", 2024-05-08T08:25:30.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4090000000000001e-05 2024-05-08T08:25:30.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.469 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.469 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.469 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:30.471 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:30.471 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:30.477+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2058 read 0x14a91d4 thru 0x14ae579 (0x53a6 bytes) 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2060 read 0x8fe12be thru 0x8fedf45 (0xcc88 bytes) 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2061 read 0x729872d thru 0x72a07ea (0x80be bytes) 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2062 read 0x2cd8adf thru 0x2cdecc8 (0x61ea bytes) 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2064 write 0x9be65b0 thru 0x9be7c44 (0x1695 bytes) 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2065 writesame 0xbbfd3ec thru 0xbbff33c data_size 0x1(0x1f51 bytes) 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2067 punch from 0x271ad32 to 0x2720ea1, (0x616f bytes) 2024-05-08T08:25:30.472 INFO:teuthology.orchestra.run.smithi031.stdout:2068 trunc from 0xbbff33d to 0x5ac6f81 2024-05-08T08:25:30.559 INFO:teuthology.orchestra.run.smithi031.stdout:2070 write 0xb590f4a thru 0xb599a22 (0x8ad9 bytes) 2024-05-08T08:25:30.563 INFO:teuthology.orchestra.run.smithi031.stdout:2072 read 0x14729e8 thru 0x14795cd (0x6be6 bytes) 2024-05-08T08:25:30.564 INFO:teuthology.orchestra.run.smithi031.stdout:2074 punch from 0x6e111d0 to 0x6e20b65, (0xf995 bytes) 2024-05-08T08:25:30.564 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.564 INFO:teuthology.orchestra.run.smithi031.stdout:2076 punch from 0x81320b5 to 0x813c0d3, (0xa01e bytes) 2024-05-08T08:25:30.564 INFO:teuthology.orchestra.run.smithi031.stdout:2078 write 0x6babc22 thru 0x6bafb01 (0x3ee0 bytes) 2024-05-08T08:25:30.566 INFO:teuthology.orchestra.run.smithi031.stdout:2079 write 0x72c0d61 thru 0x72c4c8e (0x3f2e bytes) 2024-05-08T08:25:30.568 INFO:teuthology.orchestra.run.smithi031.stdout:2080 punch from 0x665f45d to 0x66616c5, (0x2268 bytes) 2024-05-08T08:25:30.568 INFO:teuthology.orchestra.run.smithi031.stdout:2081 write 0x4d65f72 thru 0x4d6a662 (0x46f1 bytes) 2024-05-08T08:25:30.569 INFO:teuthology.orchestra.run.smithi031.stdout:2082 write 0x2e17f2a thru 0x2e27b28 (0xfbff bytes) 2024-05-08T08:25:30.571 INFO:teuthology.orchestra.run.smithi031.stdout:2083 read 0x828ec98 thru 0x828f59c (0x905 bytes) 2024-05-08T08:25:30.571 INFO:teuthology.orchestra.run.smithi031.stdout:2085 read 0xede984 thru 0xee54f0 (0x6b6d bytes) 2024-05-08T08:25:30.571 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:30.578+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:30.571 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.571 INFO:teuthology.orchestra.run.smithi031.stdout:2087 write 0x88dad4a thru 0x88dda17 (0x2cce bytes) 2024-05-08T08:25:30.581 INFO:teuthology.orchestra.run.smithi031.stdout:2088 writesame 0xbaef4d4 thru 0xbaf613d data_size 0x1(0x6c6a bytes) 2024-05-08T08:25:30.581 INFO:teuthology.orchestra.run.smithi031.stdout:2089 write 0x274c76 thru 0x279202 (0x458d bytes) 2024-05-08T08:25:30.583 INFO:teuthology.orchestra.run.smithi031.stdout:2090 punch from 0x45573af to 0x45584b7, (0x1108 bytes) 2024-05-08T08:25:30.583 INFO:teuthology.orchestra.run.smithi031.stdout:2091 punch from 0xaa6f34f to 0xaa78df5, (0x9aa6 bytes) 2024-05-08T08:25:30.589 INFO:teuthology.orchestra.run.smithi031.stdout:2092 write 0xcc4372b thru 0xcc4fcc0 (0xc596 bytes) 2024-05-08T08:25:30.593 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.593 INFO:teuthology.orchestra.run.smithi031.stdout:2098 punch from 0x4067c78 to 0x4076885, (0xec0d bytes) 2024-05-08T08:25:30.593 INFO:teuthology.orchestra.run.smithi031.stdout:2099 punch from 0x1e30279 to 0x1e3d363, (0xd0ea bytes) 2024-05-08T08:25:30.607 INFO:teuthology.orchestra.run.smithi031.stdout:2102 trunc from 0xcc4fcc1 to 0xc8738c 2024-05-08T08:25:30.619 INFO:teuthology.orchestra.run.smithi031.stdout:2103 punch from 0x72db08 to 0x72ebe0, (0x10d8 bytes) 2024-05-08T08:25:30.626 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:30.626 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:30.626 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:30.626 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:30.633 INFO:teuthology.orchestra.run.smithi031.stdout:2104 write 0x8cddc55 thru 0x8ce33cc (0x5778 bytes) 2024-05-08T08:25:30.636 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:30.636 INFO:teuthology.orchestra.run.smithi031.stdout:2105 write 0x616be91 thru 0x6179ded (0xdf5d bytes) 2024-05-08T08:25:30.638 INFO:teuthology.orchestra.run.smithi031.stdout:2107 punch from 0x2ae826f to 0x2aec186, (0x3f17 bytes) 2024-05-08T08:25:30.638 INFO:teuthology.orchestra.run.smithi031.stdout:2108 read 0xbcdfb2 thru 0xbcf26b (0x12ba bytes) 2024-05-08T08:25:30.639 INFO:teuthology.orchestra.run.smithi031.stdout:2111 read 0x438daf1 thru 0x43969fb (0x8f0b bytes) 2024-05-08T08:25:30.639 INFO:teuthology.orchestra.run.smithi031.stdout:2112 write 0x26506b thru 0x26dd01 (0x8c97 bytes) 2024-05-08T08:25:30.641 INFO:teuthology.orchestra.run.smithi031.stdout:2115 read 0x6a5a18e thru 0x6a5bafe (0x1971 bytes) 2024-05-08T08:25:30.642 INFO:teuthology.orchestra.run.smithi031.stdout:2117 read 0x38048b0 thru 0x380b12b (0x687c bytes) 2024-05-08T08:25:30.643 INFO:teuthology.orchestra.run.smithi031.stdout:2118 read 0x3ae61ce thru 0x3aef9fc (0x982f bytes) 2024-05-08T08:25:30.644 INFO:teuthology.orchestra.run.smithi031.stdout:2119 punch from 0x868bb3c to 0x86922a4, (0x6768 bytes) 2024-05-08T08:25:30.644 INFO:teuthology.orchestra.run.smithi031.stdout:2120 punch from 0x8c35f83 to 0x8c37d27, (0x1da4 bytes) 2024-05-08T08:25:30.645 INFO:teuthology.orchestra.run.smithi031.stdout:2121 write 0x218c605 thru 0x219a829 (0xe225 bytes) 2024-05-08T08:25:30.645 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.648 INFO:teuthology.orchestra.run.smithi031.stdout:2125 trunc from 0x8ce33cd to 0xb4236a2 2024-05-08T08:25:30.653 INFO:teuthology.orchestra.run.smithi031.stdout:2126 write 0xa6a62cb thru 0xa6ab16b (0x4ea1 bytes) 2024-05-08T08:25:30.662 INFO:teuthology.orchestra.run.smithi031.stdout:2127 write 0xbb26026 thru 0xbb34c78 (0xec53 bytes) 2024-05-08T08:25:30.665 INFO:teuthology.orchestra.run.smithi031.stdout:2129 punch from 0xb6cb7af to 0xb6d9993, (0xe1e4 bytes) 2024-05-08T08:25:30.666 INFO:teuthology.orchestra.run.smithi031.stdout:2130 read 0x80f84f5 thru 0x8107a40 (0xf54c bytes) 2024-05-08T08:25:30.666 INFO:teuthology.orchestra.run.smithi031.stdout:2131 writesame 0x59fecca thru 0x5a03077 data_size 0x1(0x43ae bytes) 2024-05-08T08:25:30.672 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:30.679+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:30.672 INFO:teuthology.orchestra.run.smithi031.stdout:2133 read 0x832b569 thru 0x83340c2 (0x8b5a bytes) 2024-05-08T08:25:30.676 INFO:teuthology.orchestra.run.smithi031.stdout:2134 trunc from 0xbb34c79 to 0xebc870f 2024-05-08T08:25:30.682 INFO:teuthology.orchestra.run.smithi031.stdout:2136 punch from 0xae6a28b to 0xae6d0aa, (0x2e1f bytes) 2024-05-08T08:25:30.682 INFO:teuthology.orchestra.run.smithi031.stdout:2137 read 0x5a3bf90 thru 0x5a435dd (0x764e bytes) 2024-05-08T08:25:30.682 INFO:teuthology.orchestra.run.smithi031.stdout:2138 write 0x88af2c4 thru 0x88b7fcf (0x8d0c bytes) 2024-05-08T08:25:30.685 INFO:teuthology.orchestra.run.smithi031.stdout:2139 write 0x8384584 thru 0x839101b (0xca98 bytes) 2024-05-08T08:25:30.687 INFO:teuthology.orchestra.run.smithi031.stdout:2140 write 0x553a8c4 thru 0x5546d8c (0xc4c9 bytes) 2024-05-08T08:25:30.700 INFO:teuthology.orchestra.run.smithi031.stdout:2141 trunc from 0xebc870f to 0x4458d38 2024-05-08T08:25:30.717 INFO:teuthology.orchestra.run.smithi031.stdout:2142 write 0x271bed thru 0x27df2b (0xc33f bytes) 2024-05-08T08:25:30.719 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.719 INFO:teuthology.orchestra.run.smithi031.stdout:2146 writesame 0x21f8 thru 0x39d1 data_size 0x2b(0x17da bytes) 2024-05-08T08:25:30.721 INFO:teuthology.orchestra.run.smithi031.stdout:2147 read 0x409243d thru 0x4092c0d (0x7d1 bytes) 2024-05-08T08:25:30.721 INFO:teuthology.orchestra.run.smithi031.stdout:2148 read 0x4021efc thru 0x4030eab (0xefb0 bytes) 2024-05-08T08:25:30.721 INFO:teuthology.orchestra.run.smithi031.stdout:2149 write 0x77b8e0 thru 0x786af6 (0xb217 bytes) 2024-05-08T08:25:30.735 INFO:teuthology.orchestra.run.smithi031.stdout:2151 write 0x7149ee1 thru 0x715660b (0xc72b bytes) 2024-05-08T08:25:30.737 INFO:teuthology.orchestra.run.smithi031.stdout:2152 read 0x63d0ad6 thru 0x63d163b (0xb66 bytes) 2024-05-08T08:25:30.746 INFO:teuthology.orchestra.run.smithi031.stdout:2154 write 0x930ba5e thru 0x9312f46 (0x74e9 bytes) 2024-05-08T08:25:30.748 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:30.748 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:30.750 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.751 INFO:teuthology.orchestra.run.smithi031.stdout:2156 write 0x5d714ce thru 0x5d8140c (0xff3f bytes) 2024-05-08T08:25:30.752 INFO:teuthology.orchestra.run.smithi031.stdout:2157 punch from 0x4b2baa7 to 0x4b3090c, (0x4e65 bytes) 2024-05-08T08:25:30.757 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:30.762 INFO:teuthology.orchestra.run.smithi031.stdout:2158 write 0xdf56e8b thru 0xdf5e885 (0x79fb bytes) 2024-05-08T08:25:30.767 INFO:teuthology.orchestra.run.smithi031.stdout:2159 read 0x3c76314 thru 0x3c85627 (0xf314 bytes) 2024-05-08T08:25:30.767 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.767 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.767 INFO:teuthology.orchestra.run.smithi031.stdout:2162 read 0x74aae3 thru 0x75506b (0xa589 bytes) 2024-05-08T08:25:30.768 INFO:teuthology.orchestra.run.smithi031.stdout:2164 read 0x471d478 thru 0x472922a (0xbdb3 bytes) 2024-05-08T08:25:30.768 INFO:teuthology.orchestra.run.smithi031.stdout:2165 writesame 0xcaba8db thru 0xcac699b data_size 0x47(0xc0c1 bytes) 2024-05-08T08:25:30.772 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:30.780+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:30.773 INFO:teuthology.orchestra.run.smithi031.stdout:2166 punch from 0x7c2b764 to 0x7c3219a, (0x6a36 bytes) 2024-05-08T08:25:30.773 INFO:teuthology.orchestra.run.smithi031.stdout:2167 write 0x171f03b thru 0x172ad4b (0xbd11 bytes) 2024-05-08T08:25:30.775 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.775 INFO:teuthology.orchestra.run.smithi031.stdout:2169 read 0x5ea8b36 thru 0x5eb4885 (0xbd50 bytes) 2024-05-08T08:25:30.776 INFO:teuthology.orchestra.run.smithi031.stdout:2170 write 0xa2000cd thru 0xa203297 (0x31cb bytes) 2024-05-08T08:25:30.778 INFO:teuthology.orchestra.run.smithi031.stdout:2172 write 0xa67c2f4 thru 0xa683dbe (0x7acb bytes) 2024-05-08T08:25:30.781 INFO:teuthology.orchestra.run.smithi031.stdout:2173 read 0x4e2acd0 thru 0x4e31980 (0x6cb1 bytes) 2024-05-08T08:25:30.782 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.782 INFO:teuthology.orchestra.run.smithi031.stdout:2176 read 0x438933f thru 0x439276c (0x942e bytes) 2024-05-08T08:25:30.782 INFO:teuthology.orchestra.run.smithi031.stdout:2178 read 0x45f756 thru 0x46bdfd (0xc6a8 bytes) 2024-05-08T08:25:30.783 INFO:teuthology.orchestra.run.smithi031.stdout:2179 write 0x4046aa9 thru 0x4054755 (0xdcad bytes) 2024-05-08T08:25:30.785 INFO:teuthology.orchestra.run.smithi031.stdout:2180 read 0x87aa447 thru 0x87af7f0 (0x53aa bytes) 2024-05-08T08:25:30.785 INFO:teuthology.orchestra.run.smithi031.stdout:2181 flatten 2024-05-08T08:25:30.791 INFO:teuthology.orchestra.run.smithi031.stdout:2183 read 0x734422e thru 0x7345db3 (0x1b86 bytes) 2024-05-08T08:25:30.791 INFO:teuthology.orchestra.run.smithi031.stdout:2185 read 0x7f1d5a9 thru 0x7f1f638 (0x2090 bytes) 2024-05-08T08:25:30.791 INFO:teuthology.orchestra.run.smithi031.stdout:2186 write 0x9a24a60 thru 0x9a26c76 (0x2217 bytes) 2024-05-08T08:25:30.793 INFO:teuthology.orchestra.run.smithi031.stdout:2193 write 0x8d6fc05 thru 0x8d75c49 (0x6045 bytes) 2024-05-08T08:25:30.796 INFO:teuthology.orchestra.run.smithi031.stdout:2194 write 0x9b09347 thru 0x9b17af5 (0xe7af bytes) 2024-05-08T08:25:30.798 INFO:teuthology.orchestra.run.smithi031.stdout:2195 punch from 0x5ff2c11 to 0x5ff6a96, (0x3e85 bytes) 2024-05-08T08:25:30.798 INFO:teuthology.orchestra.run.smithi031.stdout:2196 punch from 0x1bd4fcf to 0x1be3cd8, (0xed09 bytes) 2024-05-08T08:25:30.798 INFO:teuthology.orchestra.run.smithi031.stdout:2199 write 0x6f7bb32 thru 0x6f7beba (0x389 bytes) 2024-05-08T08:25:30.799 INFO:teuthology.orchestra.run.smithi031.stdout:2200 writesame 0x7375c57 thru 0x737ecae data_size 0x7c(0x9058 bytes) 2024-05-08T08:25:30.800 INFO:teuthology.orchestra.run.smithi031.stdout:2201 writesame 0xbd97c42 thru 0xbda0779 data_size 0x87(0x8b38 bytes) 2024-05-08T08:25:30.807 INFO:teuthology.orchestra.run.smithi031.stdout:2202 trunc from 0xdf5e886 to 0x911ccdd 2024-05-08T08:25:30.816 INFO:teuthology.orchestra.run.smithi031.stdout:2203 write 0x3810392 thru 0x381ec57 (0xe8c6 bytes) 2024-05-08T08:25:30.818 INFO:teuthology.orchestra.run.smithi031.stdout:2204 punch from 0x3327b04 to 0x3334bec, (0xd0e8 bytes) 2024-05-08T08:25:30.818 INFO:teuthology.orchestra.run.smithi031.stdout:2205 writesame 0x840211b thru 0x840d3d5 data_size 0x1(0xb2bb bytes) 2024-05-08T08:25:30.840 INFO:teuthology.orchestra.run.smithi031.stdout:2206 punch from 0x8342056 to 0x8348ccf, (0x6c79 bytes) 2024-05-08T08:25:30.840 INFO:teuthology.orchestra.run.smithi031.stdout:2207 write 0x1c8452c thru 0x1c8fce9 (0xb7be bytes) 2024-05-08T08:25:30.841 INFO:teuthology.orchestra.run.smithi031.stdout:2208 read 0x8e39f5c thru 0x8e3f3f8 (0x549d bytes) 2024-05-08T08:25:30.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:30.853 INFO:teuthology.orchestra.run.smithi031.stdout:2209 write 0xcfc5f6e thru 0xcfc725b (0x12ee bytes) 2024-05-08T08:25:30.854 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.855 INFO:teuthology.orchestra.run.smithi031.stdout:2214 writesame 0x5d1039e thru 0x5d1d56e data_size 0x13(0xd1d1 bytes) 2024-05-08T08:25:30.856 INFO:teuthology.orchestra.run.smithi031.stdout:2215 read 0x91567f1 thru 0x915ae48 (0x4658 bytes) 2024-05-08T08:25:30.859 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-05-08T08:25:30.860 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:25:30.860 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-08T08:25:30.860 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:25:30.862 INFO:teuthology.orchestra.run.smithi031.stdout:2216 trunc from 0xcfc725c to 0x8b33d7b 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:30.872 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 33.312698462999997, 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.875 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:88 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 33.157213058000004, 2024-05-08T08:25:30.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084184770000000006, 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 88 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721094+0000", 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1380000000000001e-06 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721098+0000", 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4120000000000001e-06 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721100+0000", 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6910000000000002e-06 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725525+0000", 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044250560000000001 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725656+0000", 2024-05-08T08:25:30.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013079000000000001 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725765+0000", 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000108571 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728094+0000", 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023291229999999998 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729456+0000", 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013616780000000001 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729487+0000", 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0956e-05 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729510+0000", 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3711999999999999e-05 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.883 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:107 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958214534, 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083640399999999997, 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 107 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.884 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920091+0000", 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920093+0000", 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.661e-06 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920098+0000", 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2560000000000004e-06 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920101+0000", 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.932e-06 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920638+0000", 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053755999999999997 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920689+0000", 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0627999999999997e-05 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920762+0000", 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2768999999999994e-05 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925616+0000", 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048541820000000003 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928394+0000", 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027782010000000001 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928431+0000", 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6915000000000001e-05 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928454+0000", 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3547999999999999e-05 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.890 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958198764000002, 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084564540000000004, 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 108 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920108+0000", 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9740000000000001e-06 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920112+0000", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920114+0000", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6630000000000002e-06 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920996+0000", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00088141399999999996 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921044+0000", 2024-05-08T08:25:30.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8078000000000003e-05 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921120+0000", 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6469000000000003e-05 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925621+0000", 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045009630000000002 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928490+0000", 2024-05-08T08:25:30.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028691929999999999 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928539+0000", 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8857000000000003e-05 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928563+0000", 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3302000000000002e-05 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958186777000002, 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085274359999999993, 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 109 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1e-07 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920144+0000", 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.586e-05 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920148+0000", 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1790000000000001e-06 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920151+0000", 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0359999999999998e-06 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921404+0000", 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00125264 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921448+0000", 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4227000000000002e-05 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921520+0000", 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1532999999999994e-05 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925631+0000", 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041116260000000002 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928593+0000", 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002961881 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928621+0000", 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7259000000000001e-05 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928646+0000", 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4984999999999999e-05 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958145852000001, 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085697380000000004, 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.901 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920162+0000", 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920165+0000", 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4640000000000002e-06 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920168+0000", 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6249999999999999e-06 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921849+0000", 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016816789999999999 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921895+0000", 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5198000000000003e-05 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921968+0000", 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3178999999999998e-05 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925638+0000", 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036703859999999999 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928677+0000", 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003038303 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928708+0000", 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1186000000000003e-05 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928729+0000", 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1035000000000001e-05 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958133038, 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086906299999999995, 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.907 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 111 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.03e-07 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920174+0000", 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9230000000000001e-06 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920177+0000", 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0539999999999999e-06 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920180+0000", 2024-05-08T08:25:30.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8480000000000001e-06 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922200+0000", 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020199060000000001 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922246+0000", 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6319999999999997e-05 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:30.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0517000000000003e-05 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925642+0000", 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033256649999999998 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928770+0000", 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031275640000000002 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928829+0000", 2024-05-08T08:25:30.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8690000000000002e-05 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928862+0000", 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3939999999999997e-05 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958121191000004, 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087670669999999999, 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.912 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 112 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920214+0000", 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9626999999999999e-05 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920218+0000", 2024-05-08T08:25:30.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1320000000000004e-06 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920225+0000", 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5859999999999999e-06 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922574+0000", 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002349009 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922651+0000", 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7014999999999994e-05 2024-05-08T08:25:30.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922793+0000", 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141839 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928736+0000", 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005942363 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928901+0000", 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00016587700000000001 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928930+0000", 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8487e-05 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928951+0000", 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0908e-05 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:30.917 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958070720000002, 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087948200000000001, 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 113 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.79e-07 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920237+0000", 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.368e-06 2024-05-08T08:25:30.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920241+0000", 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920244+0000", 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7549999999999999e-06 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923291+0000", 2024-05-08T08:25:30.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030469809999999998 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923338+0000", 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6501000000000001e-05 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923409+0000", 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1198000000000005e-05 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928739+0000", 2024-05-08T08:25:30.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053299819999999996 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928985+0000", 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00024582599999999999 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929010+0000", 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5145999999999999e-05 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929029+0000", 2024-05-08T08:25:30.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9184000000000001e-05 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958056024000001, 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088585829999999997, 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 114 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.11e-07 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920251+0000", 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0310000000000001e-06 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920254+0000", 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1549999999999999e-06 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920257+0000", 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7360000000000001e-06 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923648+0000", 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033905189999999998 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923693+0000", 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170999999999998e-05 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923764+0000", 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0875000000000002e-05 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928874+0000", 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051101599999999999 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929058+0000", 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018473 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929087+0000", 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8495000000000001e-05 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929107+0000", 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.05e-05 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.958043517, 2024-05-08T08:25:30.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089168839999999999, 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 115 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920262+0000", 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920301+0000", 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9572e-05 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920305+0000", 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2390000000000004e-06 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920309+0000", 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.14e-06 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924004+0000", 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036953369999999999 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924048+0000", 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4342999999999999e-05 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924120+0000", 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1489999999999995e-05 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928881+0000", 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047611650000000004 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929135+0000", 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025369599999999998 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929159+0000", 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4479999999999999e-05 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929178+0000", 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9208999999999999e-05 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.957990891000001, 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089378329999999992, 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 116 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6199999999999999e-07 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920316+0000", 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1260000000000001e-06 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920326+0000", 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4229999999999994e-06 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920330+0000", 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924512+0000", 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041817440000000003 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924558+0000", 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6285000000000002e-05 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924639+0000", 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0653000000000002e-05 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928884+0000", 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042451420000000004 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929204+0000", 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032002899999999999 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929232+0000", 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8048e-05 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929252+0000", 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9581000000000001e-05 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.957968592999997, 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008977779, 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.939 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 117 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920337+0000", 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7000000000000001e-07 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920341+0000", 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3039999999999998e-06 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920346+0000", 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920348+0000", 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6809999999999998e-06 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925057+0000", 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047081240000000002 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925101+0000", 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4838e-05 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925174+0000", 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2619000000000004e-05 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928959+0000", 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037846730000000001 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929272+0000", 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00031341099999999998 2024-05-08T08:25:30.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929296+0000", 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4226000000000001e-05 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929314+0000", 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7808999999999998e-05 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.944 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.261976576999999, 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083370549999999995, 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 139 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616329+0000", 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0499999999999999e-07 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616332+0000", 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8219999999999998e-06 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616336+0000", 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616339+0000", 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1650000000000002e-06 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621594+0000", 2024-05-08T08:25:30.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052553139999999996 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621649+0000", 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4416000000000002e-05 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621721+0000", 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1977000000000006e-05 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.623088+0000", 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013677889999999999 2024-05-08T08:25:30.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624604+0000", 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0015159279999999999 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624643+0000", 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8504999999999998e-05 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624665+0000", 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2543000000000001e-05 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.949 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.261958943000003, 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083691970000000001, 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.950 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 140 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.41e-07 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616374+0000", 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8e-05 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616379+0000", 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3579999999999996e-06 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616381+0000", 2024-05-08T08:25:30.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8389999999999998e-06 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621985+0000", 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056040489999999998 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622031+0000", 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5683000000000003e-05 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622098+0000", 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7063000000000005e-05 2024-05-08T08:25:30.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.623381+0000", 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001283228 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624673+0000", 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001291295 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624697+0000", 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4122999999999999e-05 2024-05-08T08:25:30.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624715+0000", 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8318e-05 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.261916833999997, 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098144059999999995, 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.955 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 141 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3299999999999998e-07 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616391+0000", 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4360000000000001e-06 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616395+0000", 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.822e-06 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616397+0000", 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.729e-06 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622404+0000", 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0060065120000000003 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622448+0000", 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3707e-05 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622520+0000", 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624923+0000", 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024033819999999999 2024-05-08T08:25:30.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626156+0000", 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001232216 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626183+0000", 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7347000000000001e-05 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626202+0000", 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9564000000000001e-05 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.960 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:6914 3.13 3:cdf44573:::rbd_data.119120fdbc10.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13546835~4324] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.722422807999999, 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0094012790000000002, 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 6914 2024-05-08T08:25:30.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155884+0000", 2024-05-08T08:25:30.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7060000000000001e-06 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159051+0000", 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031671109999999998 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159073+0000", 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1667000000000001e-05 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159083+0000", 2024-05-08T08:25:30.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0881e-05 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159151+0000", 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7706000000000005e-05 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159252+0000", 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000101098 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159508+0000", 2024-05-08T08:25:30.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025555700000000001 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.163927+0000", 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044189390000000002 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165209+0000", 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012823610000000001 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165248+0000", 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9184999999999998e-05 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165283+0000", 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5067999999999997e-05 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14412 3.d 3:b3426b33:::rbd_data.119174045df.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8540878020000005, 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024946593, 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.966 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14412 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:30.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024218+0000", 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024221+0000", 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5489999999999998e-06 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024231+0000", 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.611e-06 2024-05-08T08:25:30.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024236+0000", 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1950000000000002e-06 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024278+0000", 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2191000000000003e-05 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024328+0000", 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0008e-05 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.028903+0000", 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045743340000000002 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049023+0000", 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020120815 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049106+0000", 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2900999999999998e-05 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049140+0000", 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4029000000000002e-05 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049164+0000", 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3280000000000001e-05 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23281 3.15 3:aef679f3:::rbd_data.1191ea0f1b1f.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:30.971 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.40185470400000001, 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015244407, 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23281 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476452+0000", 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.562e-06 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483043+0000", 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065909009999999997 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483062+0000", 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9538000000000002e-05 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483074+0000", 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1915e-05 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483133+0000", 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9339000000000002e-05 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483271+0000", 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013750500000000001 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483684+0000", 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0004126 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.486766+0000", 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030822380000000002 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491572+0000", 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048065629999999998 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491643+0000", 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0518000000000005e-05 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491695+0000", 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1727999999999997e-05 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23287 3.13 3:ced44c57:::rbd_data.1191ea0f1b1f.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.36524573700000001, 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020994202999999999, 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23287 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513061+0000", 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5889999999999999e-06 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524070+0000", 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011009694 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524089+0000", 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8819000000000001e-05 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524099+0000", 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9580000000000002e-06 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:30.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524157+0000", 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8106000000000002e-05 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524271+0000", 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000113231 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524698+0000", 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00042707900000000002 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533002+0000", 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083041999999999994 2024-05-08T08:25:30.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533940+0000", 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00093861900000000002 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534001+0000", 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0492999999999998e-05 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534053+0000", 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2414999999999999e-05 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.981 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.982 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:30.982 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:30.982 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:30.984 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:30.989+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:30.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:30.986 INFO:teuthology.orchestra.run.smithi031.stdout:2217 write 0xb0e8f9d thru 0xb0ea199 (0x11fd bytes) 2024-05-08T08:25:30.986 INFO:teuthology.orchestra.run.smithi031.stdout:2219 punch from 0x98d86db to 0x98e807c, (0xf9a1 bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2220 punch from 0x8ef11eb to 0x8efc058, (0xae6d bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2221 write 0x2c4bcfb thru 0x2c519b7 (0x5cbd bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2223 read 0x7728ba1 thru 0x773332c (0xa78c bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2225 punch from 0x9545f71 to 0x9554f93, (0xf022 bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2227 write 0x6909bdc thru 0x690eb6c (0x4f91 bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2228 write 0x52a980a thru 0x52adf4f (0x4746 bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2230 read 0x14a3bd6 thru 0x14a5fbc (0x23e7 bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2231 read 0x1ac1f6c thru 0x1ad1232 (0xf2c7 bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2232 writesame 0xc73da6d thru 0xc74a2ed data_size 0x1(0xc881 bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2233 punch from 0xa715b9 to 0xa756a5, (0x40ec bytes) 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xedcc280 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2234 trunc from 0xc74a2ee to 0xedcc280 2024-05-08T08:25:30.987 INFO:teuthology.orchestra.run.smithi031.stdout:2235 trunc from 0xedcc280 to 0x80f9de0 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2236 trunc from 0x80f9de0 to 0xe8a792c 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2238 trunc from 0xe8a792c to 0x8fe30b4 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2239 writesame 0x9063e45 thru 0x9071a5c data_size 0x4(0xdc18 bytes) 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2240 trunc from 0x9071a5d to 0xec0fdcc 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2241 writesame 0x2966458 thru 0x296bead data_size 0x1f(0x5a56 bytes) 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2242 read 0x7dd6562 thru 0x7ddd040 (0x6adf bytes) 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2243 punch from 0x294abca to 0x29513b0, (0x67e6 bytes) 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2244 write 0xe9182d2 thru 0xe91e000 (0x5d2f bytes) 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2245 read 0xdf14a8a thru 0xdf17d7f (0x32f6 bytes) 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2246 write 0xe85b0e4 thru 0xe85da29 (0x2946 bytes) 2024-05-08T08:25:30.988 INFO:teuthology.orchestra.run.smithi031.stdout:2247 writesame 0xc4b4748 thru 0xc4b54a8 data_size 0x5(0xd61 bytes) 2024-05-08T08:25:30.989 INFO:teuthology.orchestra.run.smithi031.stdout:2249 write 0x8b1528 thru 0x8b9766 (0x823f bytes) 2024-05-08T08:25:30.989 INFO:teuthology.orchestra.run.smithi031.stdout:2251 write 0x652f289 thru 0x653f0b4 (0xfe2c bytes) 2024-05-08T08:25:30.991 INFO:teuthology.orchestra.run.smithi031.stdout:2252 punch from 0xc9f8941 to 0xca03be6, (0xb2a5 bytes) 2024-05-08T08:25:30.991 INFO:teuthology.orchestra.run.smithi031.stdout:2253 write 0x86fe7fb thru 0x86ff174 (0x97a bytes) 2024-05-08T08:25:30.994 INFO:teuthology.orchestra.run.smithi031.stdout:2254 writesame 0xdb8c43a thru 0xdb9bdfa data_size 0x1(0xf9c1 bytes) 2024-05-08T08:25:31.004 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:31.005 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:31.023 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:31.024 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:31.024 INFO:teuthology.orchestra.run.smithi031.stdout:2255 write 0xd159389 thru 0xd161b2d (0x87a5 bytes) 2024-05-08T08:25:31.026 INFO:teuthology.orchestra.run.smithi031.stdout:2256 write 0xd46e930 thru 0xd4708ef (0x1fc0 bytes) 2024-05-08T08:25:31.038 INFO:teuthology.orchestra.run.smithi031.stdout:2258 trunc from 0xec0fdcc to 0x7813d85 2024-05-08T08:25:31.043 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:31.043 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:31.050 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:25:31.067 INFO:teuthology.orchestra.run.smithi031.stdout:2260 write 0x9c10d69 thru 0x9c17194 (0x642c bytes) 2024-05-08T08:25:31.073 INFO:teuthology.orchestra.run.smithi031.stdout:2261 read 0x53ece thru 0x55d91 (0x1ec4 bytes) 2024-05-08T08:25:31.074 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.074 INFO:teuthology.orchestra.run.smithi031.stdout:2264 punch from 0x2876ff7 to 0x2877669, (0x672 bytes) 2024-05-08T08:25:31.074 INFO:teuthology.orchestra.run.smithi031.stdout:2266 punch from 0x447e2db to 0x448d076, (0xed9b bytes) 2024-05-08T08:25:31.080 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:31.080 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:31.084 INFO:teuthology.orchestra.run.smithi031.stdout:2267 writesame 0xdecbf92 thru 0xded6bfa data_size 0x17(0xac69 bytes) 2024-05-08T08:25:31.084 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.084 INFO:teuthology.orchestra.run.smithi031.stdout:2269 writesame 0x5b9783d thru 0x5b9b72f data_size 0xb(0x3ef3 bytes) 2024-05-08T08:25:31.084 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:31.091+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:31.087 INFO:teuthology.orchestra.run.smithi031.stdout:2270 punch from 0xda66ae3 to 0xda6e68b, (0x7ba8 bytes) 2024-05-08T08:25:31.087 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.087 INFO:teuthology.orchestra.run.smithi031.stdout:2272 write 0x6a07843 thru 0x6a0f2c4 (0x7a82 bytes) 2024-05-08T08:25:31.089 INFO:teuthology.orchestra.run.smithi031.stdout:2273 writesame 0xd67c5f4 thru 0xd67ca5b data_size 0xc(0x468 bytes) 2024-05-08T08:25:31.093 INFO:teuthology.orchestra.run.smithi031.stdout:2274 punch from 0x22316d8 to 0x223b5fe, (0x9f26 bytes) 2024-05-08T08:25:31.093 INFO:teuthology.orchestra.run.smithi031.stdout:2278 punch from 0x42ed7f0 to 0x42f8c08, (0xb418 bytes) 2024-05-08T08:25:31.093 INFO:teuthology.orchestra.run.smithi031.stdout:2280 read 0xbca99cd thru 0xbcab847 (0x1e7b bytes) 2024-05-08T08:25:31.093 INFO:teuthology.orchestra.run.smithi031.stdout:2281 writesame 0x7b005b4 thru 0x7b02522 data_size 0x1(0x1f6f bytes) 2024-05-08T08:25:31.098 INFO:teuthology.orchestra.run.smithi031.stdout:2285 write 0x74a8137 thru 0x74adaa5 (0x596f bytes) 2024-05-08T08:25:31.099 INFO:teuthology.orchestra.run.smithi031.stdout:2288 read 0x956d699 thru 0x956fe87 (0x27ef bytes) 2024-05-08T08:25:31.100 INFO:teuthology.orchestra.run.smithi031.stdout:2289 punch from 0xa29be7d to 0xa2a3c1b, (0x7d9e bytes) 2024-05-08T08:25:31.100 INFO:teuthology.orchestra.run.smithi031.stdout:2291 read 0x8974b70 thru 0x897fa9c (0xaf2d bytes) 2024-05-08T08:25:31.100 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.100 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.100 INFO:teuthology.orchestra.run.smithi031.stdout:2294 read 0x4e640ee thru 0x4e70dd7 (0xccea bytes) 2024-05-08T08:25:31.101 INFO:teuthology.orchestra.run.smithi031.stdout:2295 read 0x62c95c3 thru 0x62cddd3 (0x4811 bytes) 2024-05-08T08:25:31.101 INFO:teuthology.orchestra.run.smithi031.stdout:2296 write 0xf4caf7 thru 0xf56b58 (0xa062 bytes) 2024-05-08T08:25:31.103 INFO:teuthology.orchestra.run.smithi031.stdout:2298 read 0x432fb37 thru 0x433f170 (0xf63a bytes) 2024-05-08T08:25:31.103 INFO:teuthology.orchestra.run.smithi031.stdout:2299 writesame 0x7d65b58 thru 0x7d68143 data_size 0x6(0x25ec bytes) 2024-05-08T08:25:31.105 INFO:teuthology.orchestra.run.smithi031.stdout:2302 read 0x3976a0 thru 0x3982ec (0xc4d bytes) 2024-05-08T08:25:31.106 INFO:teuthology.orchestra.run.smithi031.stdout:2303 write 0xb9a4dbd thru 0xb9ad81a (0x8a5e bytes) 2024-05-08T08:25:31.110 INFO:teuthology.orchestra.run.smithi031.stdout:2304 write 0x27abd6d thru 0x27ba1bf (0xe453 bytes) 2024-05-08T08:25:31.112 INFO:teuthology.orchestra.run.smithi031.stdout:2305 write 0x26ffb61 thru 0x270a3ac (0xa84c bytes) 2024-05-08T08:25:31.115 INFO:teuthology.orchestra.run.smithi031.stdout:2306 write 0x51dfcd6 thru 0x51eb75a (0xba85 bytes) 2024-05-08T08:25:31.116 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:31.116 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:31.119 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:31.119 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:31.127 INFO:teuthology.orchestra.run.smithi031.stdout:2308 trunc from 0xded6bfb to 0x46227f9 2024-05-08T08:25:31.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:31.140 INFO:teuthology.orchestra.run.smithi031.stdout:2309 read 0x1b54845 thru 0x1b62d63 (0xe51f bytes) 2024-05-08T08:25:31.142 INFO:teuthology.orchestra.run.smithi031.stdout:2310 punch from 0x442dca0 to 0x4431f4a, (0x42aa bytes) 2024-05-08T08:25:31.142 INFO:teuthology.orchestra.run.smithi031.stdout:2311 read 0x26549bf thru 0x2660834 (0xbe76 bytes) 2024-05-08T08:25:31.142 INFO:teuthology.orchestra.run.smithi031.stdout:2312 punch from 0x25a9435 to 0x25b2a1a, (0x95e5 bytes) 2024-05-08T08:25:31.142 INFO:teuthology.orchestra.run.smithi031.stdout:2314 punch from 0xdb5b8f to 0xdc29c0, (0xce31 bytes) 2024-05-08T08:25:31.143 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.143 INFO:teuthology.orchestra.run.smithi031.stdout:2317 read 0xc24a98 thru 0xc257cf (0xd38 bytes) 2024-05-08T08:25:31.145 INFO:teuthology.orchestra.run.smithi031.stdout:2318 trunc from 0x46227f9 to 0x21b45bb 2024-05-08T08:25:31.154 INFO:teuthology.orchestra.run.smithi031.stdout:2319 punch from 0x1bbf7c6 to 0x1bc678d, (0x6fc7 bytes) 2024-05-08T08:25:31.155 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:31.155 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:31.155 INFO:teuthology.orchestra.run.smithi031.stdout:2320 trunc from 0x21b45bb to 0x2cae6f3 2024-05-08T08:25:31.160 INFO:teuthology.orchestra.run.smithi031.stdout:2321 read 0x1e442a8 thru 0x1e52658 (0xe3b1 bytes) 2024-05-08T08:25:31.160 INFO:teuthology.orchestra.run.smithi031.stdout:2322 read 0x1354a62 thru 0x1357137 (0x26d6 bytes) 2024-05-08T08:25:31.161 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.175 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:31.175 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:31.175 INFO:teuthology.orchestra.run.smithi031.stdout:2324 write 0x963c791 thru 0x9641d1d (0x558d bytes) 2024-05-08T08:25:31.179 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:31.179 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:31.182 INFO:teuthology.orchestra.run.smithi031.stdout:2326 trunc from 0x9641d1e to 0x6583c68 2024-05-08T08:25:31.184 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:31.191+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:31.200 INFO:teuthology.orchestra.run.smithi031.stdout:2327 write 0x7e27f2f thru 0x7e341a8 (0xc27a bytes) 2024-05-08T08:25:31.201 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.207 INFO:teuthology.orchestra.run.smithi031.stdout:2331 trunc from 0x7e341a9 to 0xd7c3207 2024-05-08T08:25:31.213 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.213 INFO:teuthology.orchestra.run.smithi031.stdout:2333 read 0xa24a8c7 thru 0xa253c1d (0x9357 bytes) 2024-05-08T08:25:31.213 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.213 INFO:teuthology.orchestra.run.smithi031.stdout:2335 write 0x8cb3b8 thru 0x8cd8a0 (0x24e9 bytes) 2024-05-08T08:25:31.215 INFO:teuthology.orchestra.run.smithi031.stdout:2336 punch from 0x49c9f57 to 0x49cb6df, (0x1788 bytes) 2024-05-08T08:25:31.215 INFO:teuthology.orchestra.run.smithi031.stdout:2337 read 0x50f68ff thru 0x51011c8 (0xa8ca bytes) 2024-05-08T08:25:31.215 INFO:teuthology.orchestra.run.smithi031.stdout:2338 read 0x3eff9b2 thru 0x3f0b9ea (0xc039 bytes) 2024-05-08T08:25:31.216 INFO:teuthology.orchestra.run.smithi031.stdout:2339 read 0x8235828 thru 0x823bbe7 (0x63c0 bytes) 2024-05-08T08:25:31.216 INFO:teuthology.orchestra.run.smithi031.stdout:2342 punch from 0xb3b08b8 to 0xb3be8b4, (0xdffc bytes) 2024-05-08T08:25:31.216 INFO:teuthology.orchestra.run.smithi031.stdout:2343 writesame 0x5da6d4f thru 0x5db61f7 data_size 0x1(0xf4a9 bytes) 2024-05-08T08:25:31.236 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.236 INFO:teuthology.orchestra.run.smithi031.stdout:2345 read 0x9df7e8f thru 0x9dfeb76 (0x6ce8 bytes) 2024-05-08T08:25:31.237 INFO:teuthology.orchestra.run.smithi031.stdout:2346 write 0x754b6e5 thru 0x75513b1 (0x5ccd bytes) 2024-05-08T08:25:31.238 INFO:teuthology.orchestra.run.smithi031.stdout:2348 read 0x9145936 thru 0x9146737 (0xe02 bytes) 2024-05-08T08:25:31.244 INFO:teuthology.orchestra.run.smithi031.stdout:2349 write 0xdf4479c thru 0xdf499aa (0x520f bytes) 2024-05-08T08:25:31.247 INFO:teuthology.orchestra.run.smithi031.stdout:2351 write 0x167883f thru 0x167a8f5 (0x20b7 bytes) 2024-05-08T08:25:31.252 INFO:teuthology.orchestra.run.smithi031.stdout:2352 trunc from 0xdf499ab to 0xb2d7b55 2024-05-08T08:25:31.258 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.258 INFO:teuthology.orchestra.run.smithi031.stdout:2355 read 0x7729fbf thru 0x773917b (0xf1bd bytes) 2024-05-08T08:25:31.259 INFO:teuthology.orchestra.run.smithi031.stdout:2356 read 0x1883685 thru 0x1888ce2 (0x565e bytes) 2024-05-08T08:25:31.259 INFO:teuthology.orchestra.run.smithi031.stdout:2358 read 0x51e6a9b thru 0x51edc47 (0x71ad bytes) 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 42, 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf", 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-05-08T08:24:44.143941+0000", 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-05-08T08:25:31.053840+0000", 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-05-08T08:25:19.984980+0000", 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-05-08T08:24:46.429925+0000", 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-05-08T08:25:31.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 4, 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:25:31.266 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:55.134472+0000", 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:25:31.267 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:25:31.268 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "16", 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:25:31.269 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:25:31.270 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:25:31.271 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 2.9900000095367432, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 2.9900000095367432, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2.630000114440918, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2.630000114440918, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:56.465515+0000", 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:25:31.272 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:25:31.273 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:25:31.274 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:25:31.275 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-05-08T08:25:31.276 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 7.690000057220459, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 7.690000057220459, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.12999999523162842, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-05-08T08:25:31.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:25:03.054989+0000", 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:25:31.278 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:25:31.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "41", 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 15, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 41, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:25:31.280 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:25:31.281 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.7400000095367432, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.7400000095367432, 2024-05-08T08:25:31.282 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.87999999523162842, 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.5299999713897705, 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.5299999713897705, 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 1, 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 1 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14afea42-356a-4163-ac3f-2e270e807917", 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:31.283 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.285 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.286 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:25:31.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/727789107", 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/727789107", 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/727789107", 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/727789107", 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "db77a923-8f49-4b47-b752-00b475917a0b", 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 0, 2024-05-08T08:25:31.288 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 32, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.290 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.291 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3462314176 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/3462314176", 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/3462314176", 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/3462314176", 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/3462314176", 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "exists" 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "3d508691-8e40-4592-82d0-eaf4c4c7d7dd", 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.293 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.295 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.296 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.297 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2736372115", 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2736372115", 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2736372115", 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2736372115", 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2545300a-428a-41ed-b19a-3a0cb9e5e755", 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.298 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.300 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:25:31.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/573482428", 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/573482428", 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/573482428", 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/573482428", 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "886aeed5-4495-4064-8579-050d38a4ad1b", 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:31.303 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6801", 2024-05-08T08:25:31.304 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6803", 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6805", 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6807", 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6813", 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6815", 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6809", 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6811", 2024-05-08T08:25:31.307 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6803/3424463091", 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6807/3424463091", 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6815/3424463091", 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6811/3424463091", 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:25:31.308 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4d15cef0-9ee9-4069-a6c4-a650e873ee03", 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6816", 2024-05-08T08:25:31.309 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6818", 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6820", 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6822", 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6828", 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.311 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6830", 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6824", 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.312 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6826", 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6818/3709461555", 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6822/3709461555", 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6830/3709461555", 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6826/3709461555", 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:25:31.313 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "910ac664-9573-4851-9e2b-89ac6015fdc1", 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.314 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6800", 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6802", 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6804", 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6806", 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6812", 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6814", 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6808", 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.317 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6810", 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6802/405561966", 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6806/405561966", 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6814/405561966", 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6810/405561966", 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:25:31.318 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14cc2d94-48e4-47c9-896a-9be9281ef0c5", 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.319 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6817", 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6819", 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6821", 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6823", 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6829", 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6831", 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6825", 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:25:31.322 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6827", 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6819/570236217", 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6823/570236217", 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6831/570236217", 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6827/570236217", 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:31.323 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.572981+0000", 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.324 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.626543+0000", 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 32 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.638878+0000", 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:31.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.599293+0000", 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.326 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.565676+0000", 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.552124+0000", 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.531428+0000", 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:25:31.328 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.658391+0000", 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-05-08T08:25:31.329 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:25:31.330 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-05-08T08:25:31.331 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:31.331 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:31.331 INFO:teuthology.orchestra.run.smithi031.stdout:2361 trunc from 0xb2d7b55 to 0x749b96e 2024-05-08T08:25:31.331 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:31.331 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:2362 read 0x2cbe916 thru 0x2cc8831 (0x9f1c bytes) 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:2367 read 0x5728b92 thru 0x5731756 (0x8bc5 bytes) 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:2369 read 0x5fe5a8 thru 0x600450 (0x1ea9 bytes) 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:2370 punch from 0x57a25b3 to 0x57abfce, (0x9a1b bytes) 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:2371 write 0x4234922 thru 0x4236444 (0x1b23 bytes) 2024-05-08T08:25:31.332 INFO:teuthology.orchestra.run.smithi031.stdout:2375 read 0x32df955 thru 0x32eb6be (0xbd6a bytes) 2024-05-08T08:25:31.333 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:31.335 INFO:tasks.thrashosds.thrasher:j is {'epoch': 42, 'fsid': '1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf', 'created': '2024-05-08T08:24:44.143941+0000', 'modified': '2024-05-08T08:25:31.053840+0000', 'last_up_change': '2024-05-08T08:25:19.984980+0000', 'last_in_change': '2024-05-08T08:24:46.429925+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:24:55.134472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.990000009536743, 'score_stable': 2.990000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:24:56.465515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.690000057220459, 'score_stable': 7.690000057220459, 'optimal_score': 0.12999999523162842, 'raw_score_acting': 1, 'raw_score_stable': 1, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:25:03.054989+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 41, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '14afea42-356a-4163-ac3f-2e270e807917', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 727789107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 727789107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 727789107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 727789107}]}, 'public_addr': '172.21.15.31:6801/727789107', 'cluster_addr': '172.21.15.31:6803/727789107', 'heartbeat_back_addr': '172.21.15.31:6807/727789107', 'heartbeat_front_addr': '172.21.15.31:6805/727789107', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'db77a923-8f49-4b47-b752-00b475917a0b', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 32, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 3462314176}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 3462314176}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 3462314176}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 3462314176}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 3462314176}]}, 'public_addr': '172.21.15.31:6817/3462314176', 'cluster_addr': '172.21.15.31:6819/3462314176', 'heartbeat_back_addr': '172.21.15.31:6823/3462314176', 'heartbeat_front_addr': '172.21.15.31:6821/3462314176', 'state': ['exists']}, {'osd': 2, 'uuid': '3d508691-8e40-4592-82d0-eaf4c4c7d7dd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2736372115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2736372115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2736372115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2736372115}]}, 'public_addr': '172.21.15.31:6809/2736372115', 'cluster_addr': '172.21.15.31:6811/2736372115', 'heartbeat_back_addr': '172.21.15.31:6815/2736372115', 'heartbeat_front_addr': '172.21.15.31:6813/2736372115', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2545300a-428a-41ed-b19a-3a0cb9e5e755', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 573482428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 573482428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 573482428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 573482428}]}, 'public_addr': '172.21.15.31:6825/573482428', 'cluster_addr': '172.21.15.31:6827/573482428', 'heartbeat_back_addr': '172.21.15.31:6831/573482428', 'heartbeat_front_addr': '172.21.15.31:6829/573482428', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '886aeed5-4495-4064-8579-050d38a4ad1b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6801', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6803', 'nonce': 3424463091}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6805', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6807', 'nonce': 3424463091}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6813', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6815', 'nonce': 3424463091}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6809', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6811', 'nonce': 3424463091}]}, 'public_addr': '172.21.15.149:6803/3424463091', 'cluster_addr': '172.21.15.149:6807/3424463091', 'heartbeat_back_addr': '172.21.15.149:6815/3424463091', 'heartbeat_front_addr': '172.21.15.149:6811/3424463091', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d15cef0-9ee9-4069-a6c4-a650e873ee03', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6816', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6818', 'nonce': 3709461555}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6820', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6822', 'nonce': 3709461555}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6828', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6830', 'nonce': 3709461555}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6824', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6826', 'nonce': 3709461555}]}, 'public_addr': '172.21.15.149:6818/3709461555', 'cluster_addr': '172.21.15.149:6822/3709461555', 'heartbeat_back_addr': '172.21.15.149:6830/3709461555', 'heartbeat_front_addr': '172.21.15.149:6826/3709461555', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '910ac664-9573-4851-9e2b-89ac6015fdc1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6800', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6802', 'nonce': 405561966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6804', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6806', 'nonce': 405561966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6812', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6814', 'nonce': 405561966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6808', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6810', 'nonce': 405561966}]}, 'public_addr': '172.21.15.149:6802/405561966', 'cluster_addr': '172.21.15.149:6806/405561966', 'heartbeat_back_addr': '172.21.15.149:6814/405561966', 'heartbeat_front_addr': '172.21.15.149:6810/405561966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '14cc2d94-48e4-47c9-896a-9be9281ef0c5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6817', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6819', 'nonce': 570236217}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6821', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6823', 'nonce': 570236217}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6829', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6831', 'nonce': 570236217}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6825', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6827', 'nonce': 570236217}]}, 'public_addr': '172.21.15.149:6819/570236217', 'cluster_addr': '172.21.15.149:6823/570236217', 'heartbeat_back_addr': '172.21.15.149:6831/570236217', 'heartbeat_front_addr': '172.21.15.149:6827/570236217', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.572981+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.626543+0000', 'dead_epoch': 32}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.638878+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.599293+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.565676+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.552124+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.531428+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.658391+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:25:31.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:31.336 INFO:teuthology.orchestra.run.smithi031.stdout:2376 read 0x108d105 thru 0x108dc94 (0xb90 bytes) 2024-05-08T08:25:31.336 INFO:teuthology.orchestra.run.smithi031.stdout:2377 write 0x6998ab9 thru 0x69a0945 (0x7e8d bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2379 read 0x3dfa3d9 thru 0x3e06b3d (0xc765 bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2380 trunc from 0x749b96e to 0x52542da 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2381 read 0x3844567 thru 0x384b0b6 (0x6b50 bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2383 write 0x910c333 thru 0x910ca7a (0x748 bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2386 punch from 0x14ff6fa to 0x150664e, (0x6f54 bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2389 read 0x70bc1b8 thru 0x70c1dd3 (0x5c1c bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2390 read 0x51ebaeb thru 0x51ebc6a (0x180 bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2393 read 0x6dc5e2a thru 0x6dcde58 (0x802f bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2394 writesame 0xa0fec51 thru 0xa0ff9c6 data_size 0x1(0xd76 bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2396 read 0x34127ec thru 0x341a5a2 (0x7db7 bytes) 2024-05-08T08:25:31.337 INFO:teuthology.orchestra.run.smithi031.stdout:2398 trunc from 0xa0ff9c7 to 0x2bf0aa1 2024-05-08T08:25:31.338 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:31.341+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:31.341 INFO:teuthology.orchestra.run.smithi031.stdout:2399 read 0x245849b thru 0x245b855 (0x33bb bytes) 2024-05-08T08:25:31.351 INFO:teuthology.orchestra.run.smithi031.stdout:2400 write 0x4cc30ee thru 0x4cd3034 (0xff47 bytes) 2024-05-08T08:25:31.352 INFO:teuthology.orchestra.run.smithi031.stdout:2402 read 0x142b6dd thru 0x142f05d (0x3981 bytes) 2024-05-08T08:25:31.353 INFO:teuthology.orchestra.run.smithi031.stdout:2403 read 0x12981e thru 0x134c0a (0xb3ed bytes) 2024-05-08T08:25:31.354 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.354 INFO:teuthology.orchestra.run.smithi031.stdout:2405 read 0x2399b64 thru 0x23a48ef (0xad8c bytes) 2024-05-08T08:25:31.355 INFO:teuthology.orchestra.run.smithi031.stdout:2406 punch from 0xaca44f to 0xad463d, (0xa1ee bytes) 2024-05-08T08:25:31.367 INFO:teuthology.orchestra.run.smithi031.stdout:2407 write 0xa717fa8 thru 0xa71fa36 (0x7a8f bytes) 2024-05-08T08:25:31.378 INFO:teuthology.orchestra.run.smithi031.stdout:2408 write 0xb446e92 thru 0xb449e50 (0x2fbf bytes) 2024-05-08T08:25:31.381 INFO:teuthology.orchestra.run.smithi031.stdout:2410 read 0x136c10b thru 0x136f2d3 (0x31c9 bytes) 2024-05-08T08:25:31.381 INFO:teuthology.orchestra.run.smithi031.stdout:2411 writesame 0x6bb822 thru 0x6c02d3 data_size 0x3(0x4ab2 bytes) 2024-05-08T08:25:31.385 INFO:teuthology.orchestra.run.smithi031.stdout:2413 read 0x84aefac thru 0x84b73e2 (0x8437 bytes) 2024-05-08T08:25:31.385 INFO:teuthology.orchestra.run.smithi031.stdout:2415 write 0x51c2096 thru 0x51cf617 (0xd582 bytes) 2024-05-08T08:25:31.394 INFO:teuthology.orchestra.run.smithi031.stdout:2416 trunc from 0xb449e51 to 0x5d811d3 2024-05-08T08:25:31.416 INFO:teuthology.orchestra.run.smithi031.stdout:2417 writesame 0x88cc852 thru 0x88d615d data_size 0x1e(0x990c bytes) 2024-05-08T08:25:31.430 INFO:teuthology.orchestra.run.smithi031.stdout:2418 writesame 0xe42f3e6 thru 0xe435f37 data_size 0x39(0x6b52 bytes) 2024-05-08T08:25:31.430 INFO:teuthology.orchestra.run.smithi031.stdout:2420 write 0xde07bb4 thru 0xde0ab3d (0x2f8a bytes) 2024-05-08T08:25:31.432 INFO:teuthology.orchestra.run.smithi031.stdout:2422 writesame 0x7f6ca9a thru 0x7f7a58d data_size 0x36(0xdaf4 bytes) 2024-05-08T08:25:31.436 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:31.442+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:31.442 INFO:teuthology.orchestra.run.smithi031.stdout:2424 trunc from 0xe435f38 to 0x74e526a 2024-05-08T08:25:31.464 INFO:teuthology.orchestra.run.smithi031.stdout:2425 write 0xaa27cce thru 0xaa28622 (0x955 bytes) 2024-05-08T08:25:31.478 INFO:teuthology.orchestra.run.smithi031.stdout:2427 trunc from 0xaa28623 to 0x1d65f28 2024-05-08T08:25:31.482 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:31.482 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:31.496 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:31.507 INFO:teuthology.orchestra.run.smithi031.stdout:2429 write 0xb2ba304 thru 0xb2c6b06 (0xc803 bytes) 2024-05-08T08:25:31.514 INFO:teuthology.orchestra.run.smithi031.stdout:2430 trunc from 0xb2c6b07 to 0xe401bdf 2024-05-08T08:25:31.520 INFO:teuthology.orchestra.run.smithi031.stdout:2434 write 0xbaea0a9 thru 0xbaf0d22 (0x6c7a bytes) 2024-05-08T08:25:31.521 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.522 INFO:teuthology.orchestra.run.smithi031.stdout:2440 write 0x237975c thru 0x23843ca (0xac6f bytes) 2024-05-08T08:25:31.524 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.524 INFO:teuthology.orchestra.run.smithi031.stdout:2443 write 0xd9dded1 thru 0xd9e5e7e (0x7fae bytes) 2024-05-08T08:25:31.528 INFO:teuthology.orchestra.run.smithi031.stdout:2444 read 0x9f79c33 thru 0x9f81c93 (0x8061 bytes) 2024-05-08T08:25:31.528 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.528 INFO:teuthology.orchestra.run.smithi031.stdout:2446 punch from 0x689b164 to 0x68aa1c0, (0xf05c bytes) 2024-05-08T08:25:31.532 INFO:teuthology.orchestra.run.smithi031.stdout:2447 trunc from 0xe401bdf to 0xa6f3123 2024-05-08T08:25:31.536 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:31.544+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:31.540 INFO:teuthology.orchestra.run.smithi031.stdout:2448 write 0x4b1cb40 thru 0x4b2a3e6 (0xd8a7 bytes) 2024-05-08T08:25:31.541 INFO:teuthology.orchestra.run.smithi031.stdout:2449 write 0x39c3959 thru 0x39c9615 (0x5cbd bytes) 2024-05-08T08:25:31.543 INFO:teuthology.orchestra.run.smithi031.stdout:2450 write 0x518b39 thru 0x526992 (0xde5a bytes) 2024-05-08T08:25:31.544 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.554 INFO:teuthology.orchestra.run.smithi031.stdout:2455 write 0xe345a54 thru 0xe34e6d4 (0x8c81 bytes) 2024-05-08T08:25:31.558 INFO:teuthology.orchestra.run.smithi031.stdout:2457 write 0xcd6ef4a thru 0xcd7063e (0x16f5 bytes) 2024-05-08T08:25:31.565 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.565 INFO:teuthology.orchestra.run.smithi031.stdout:2460 read 0xc3cd925 thru 0xc3dced6 (0xf5b2 bytes) 2024-05-08T08:25:31.565 INFO:teuthology.orchestra.run.smithi031.stdout:2462 write 0x72119b2 thru 0x721cec0 (0xb50f bytes) 2024-05-08T08:25:31.567 INFO:teuthology.orchestra.run.smithi031.stdout:2464 writesame 0x7bf2591 thru 0x7bf5092 data_size 0xf(0x2b02 bytes) 2024-05-08T08:25:31.569 INFO:teuthology.orchestra.run.smithi031.stdout:2465 punch from 0x1198431 to 0x119b5ce, (0x319d bytes) 2024-05-08T08:25:31.584 INFO:teuthology.orchestra.run.smithi031.stdout:2466 trunc from 0xe34e6d5 to 0x2321a4c 2024-05-08T08:25:31.597 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:31.597 INFO:teuthology.orchestra.run.smithi031.stdout:2470 read 0x1f1e6bf thru 0x1f2bd54 (0xd696 bytes) 2024-05-08T08:25:31.606 INFO:teuthology.orchestra.run.smithi031.stdout:2471 write 0x416f9ea thru 0x4178b09 (0x9120 bytes) 2024-05-08T08:25:31.607 INFO:teuthology.orchestra.run.smithi031.stdout:2472 read 0x36a46ac thru 0x36af2a7 (0xabfc bytes) 2024-05-08T08:25:31.617 INFO:teuthology.orchestra.run.smithi031.stdout:2473 writesame 0x70aea4c thru 0x70b9ffe data_size 0x23(0xb5b3 bytes) 2024-05-08T08:25:31.618 INFO:teuthology.orchestra.run.smithi031.stdout:2474 write 0xd02cb5 thru 0xd08a6b (0x5db7 bytes) 2024-05-08T08:25:31.619 INFO:teuthology.orchestra.run.smithi031.stdout:2476 write 0x4f54fc thru 0x501206 (0xbd0b bytes) 2024-05-08T08:25:31.621 INFO:teuthology.orchestra.run.smithi031.stdout:2477 clone 14 order 21 su 1048576 sc 3 2024-05-08T08:25:31.632 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:31.633 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:31.638 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:31.645+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:31.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:31.692 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:31.692 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:31.704 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":53,"stamp":"2024-05-08T08:25:31.059791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":580767551,"num_objects":608,"num_object_clones":112,"num_object_copies":1216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":608,"num_whiteouts":289,"num_read":35793,"num_read_kb":270640,"num_write":7275,"num_write_kb":150695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5686,"ondisk_log_size":5686,"up":74,"acting":74,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":289116,"kb_used_data":22960,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754685604,"statfs":{"total":773094113280,"available":772798058496,"internally_reserved":0,"allocated":23511040,"data_stored":199137765,"data_compressed":5222222,"data_compressed_allocated":16461824,"data_compressed_original":195373124,"omap_allocated":12749,"internal_metadata":272354867},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141371014,"num_objects":77,"num_object_clones":1,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":0,"num_read":10665,"num_read_kb":55115,"num_write":1954,"num_write_kb":35446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.022730"},"pg_stats":[{"pgid":"3.1f","version":"40'190","reported_seq":1077,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.253143+0000","last_change":"2024-05-08T08:25:06.628835+0000","last_active":"2024-05-08T08:25:30.253143+0000","last_peered":"2024-05-08T08:25:30.253143+0000","last_clean":"2024-05-08T08:25:30.253143+0000","last_became_active":"2024-05-08T08:25:04.610836+0000","last_became_peered":"2024-05-08T08:25:04.610836+0000","last_unstale":"2024-05-08T08:25:30.253143+0000","last_undegraded":"2024-05-08T08:25:30.253143+0000","last_fullsized":"2024-05-08T08:25:30.253143+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:34.215305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011177400000000001,"stat_sum":{"num_bytes":16649248,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":12,"num_read":2062,"num_read_kb":8902,"num_write":179,"num_write_kb":3477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"39'127","reported_seq":437,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.256530+0000","last_change":"2024-05-08T08:25:06.611278+0000","last_active":"2024-05-08T08:25:30.256530+0000","last_peered":"2024-05-08T08:25:30.256530+0000","last_clean":"2024-05-08T08:25:30.256530+0000","last_became_active":"2024-05-08T08:25:04.611994+0000","last_became_peered":"2024-05-08T08:25:04.611994+0000","last_unstale":"2024-05-08T08:25:30.256530+0000","last_undegraded":"2024-05-08T08:25:30.256530+0000","last_fullsized":"2024-05-08T08:25:30.256530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":127,"log_dups_size":0,"ondisk_log_size":127,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:41.431793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010168530000000001,"stat_sum":{"num_bytes":8757907,"num_objects":25,"num_object_clones":8,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":12,"num_read":362,"num_read_kb":8550,"num_write":148,"num_write_kb":9063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"39'144","reported_seq":625,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.259210+0000","last_change":"2024-05-08T08:25:06.625063+0000","last_active":"2024-05-08T08:25:30.259210+0000","last_peered":"2024-05-08T08:25:30.259210+0000","last_clean":"2024-05-08T08:25:30.259210+0000","last_became_active":"2024-05-08T08:25:04.610134+0000","last_became_peered":"2024-05-08T08:25:04.610134+0000","last_unstale":"2024-05-08T08:25:30.259210+0000","last_undegraded":"2024-05-08T08:25:30.259210+0000","last_fullsized":"2024-05-08T08:25:30.259210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:52:58.774472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005171,"stat_sum":{"num_bytes":2770944,"num_objects":18,"num_object_clones":4,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":6,"num_read":868,"num_read_kb":3541,"num_write":149,"num_write_kb":1143,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"41'121","reported_seq":1111,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.263209+0000","last_change":"2024-05-08T08:25:06.625044+0000","last_active":"2024-05-08T08:25:30.263209+0000","last_peered":"2024-05-08T08:25:30.263209+0000","last_clean":"2024-05-08T08:25:30.263209+0000","last_became_active":"2024-05-08T08:25:04.615120+0000","last_became_peered":"2024-05-08T08:25:04.615120+0000","last_unstale":"2024-05-08T08:25:30.263209+0000","last_undegraded":"2024-05-08T08:25:30.263209+0000","last_fullsized":"2024-05-08T08:25:30.263209+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:49.697724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010445210000000001,"stat_sum":{"num_bytes":83273908,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":5,"num_read":1279,"num_read_kb":22486,"num_write":179,"num_write_kb":12394,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"38'706","reported_seq":835,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.080602+0000","last_change":"2024-05-08T08:25:06.628198+0000","last_active":"2024-05-08T08:25:30.080602+0000","last_peered":"2024-05-08T08:25:30.080602+0000","last_clean":"2024-05-08T08:25:30.080602+0000","last_became_active":"2024-05-08T08:25:04.612215+0000","last_became_peered":"2024-05-08T08:25:04.612215+0000","last_unstale":"2024-05-08T08:25:30.080602+0000","last_undegraded":"2024-05-08T08:25:30.080602+0000","last_fullsized":"2024-05-08T08:25:30.080602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:06:00.305451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064530100000000003,"stat_sum":{"num_bytes":16772005,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":14,"num_read":3423,"num_read_kb":3514,"num_write":1215,"num_write_kb":1675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"39'118","reported_seq":1238,"reported_epoch":41,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:25:30.428081+0000","last_change":"2024-05-08T08:25:21.004170+0000","last_active":"2024-05-08T08:25:30.428081+0000","last_peered":"2024-05-08T08:25:30.428081+0000","last_clean":"2024-05-08T08:25:19.999438+0000","last_became_active":"2024-05-08T08:25:21.004170+0000","last_became_peered":"2024-05-08T08:25:21.004170+0000","last_unstale":"2024-05-08T08:25:30.428081+0000","last_undegraded":"2024-05-08T08:25:21.003746+0000","last_fullsized":"2024-05-08T08:25:21.003657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":118,"log_dups_size":0,"ondisk_log_size":118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:34.060497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080728699999999996,"stat_sum":{"num_bytes":24547017,"num_objects":21,"num_object_clones":2,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":1074,"num_read_kb":5431,"num_write":125,"num_write_kb":3705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":21}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"39'138","reported_seq":602,"reported_epoch":41,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:25:30.255309+0000","last_change":"2024-05-08T08:25:20.999789+0000","last_active":"2024-05-08T08:25:30.255309+0000","last_peered":"2024-05-08T08:25:30.255309+0000","last_clean":"2024-05-08T08:25:18.944546+0000","last_became_active":"2024-05-08T08:25:20.999789+0000","last_became_peered":"2024-05-08T08:25:20.999789+0000","last_unstale":"2024-05-08T08:25:30.255309+0000","last_undegraded":"2024-05-08T08:25:20.999068+0000","last_fullsized":"2024-05-08T08:25:20.998988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:52:04.276028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040135600000000001,"stat_sum":{"num_bytes":1837380,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":8,"num_read":1028,"num_read_kb":2366,"num_write":138,"num_write_kb":521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":17}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"41'124","reported_seq":597,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.282510+0000","last_change":"2024-05-08T08:25:06.610175+0000","last_active":"2024-05-08T08:25:30.282510+0000","last_peered":"2024-05-08T08:25:30.282510+0000","last_clean":"2024-05-08T08:25:30.282510+0000","last_became_active":"2024-05-08T08:25:04.609185+0000","last_became_peered":"2024-05-08T08:25:04.609185+0000","last_unstale":"2024-05-08T08:25:30.282510+0000","last_undegraded":"2024-05-08T08:25:30.282510+0000","last_fullsized":"2024-05-08T08:25:30.282510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:12:17.373611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505687,"stat_sum":{"num_bytes":6118501,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":6,"num_read":845,"num_read_kb":5053,"num_write":119,"num_write_kb":7487,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"39'411","reported_seq":1115,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.271992+0000","last_change":"2024-05-08T08:25:06.628387+0000","last_active":"2024-05-08T08:25:30.271992+0000","last_peered":"2024-05-08T08:25:30.271992+0000","last_clean":"2024-05-08T08:25:30.271992+0000","last_became_active":"2024-05-08T08:25:04.610818+0000","last_became_peered":"2024-05-08T08:25:04.610818+0000","last_unstale":"2024-05-08T08:25:30.271992+0000","last_undegraded":"2024-05-08T08:25:30.271992+0000","last_fullsized":"2024-05-08T08:25:30.271992+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":411,"log_dups_size":0,"ondisk_log_size":411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:02.932411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069309899999999997,"stat_sum":{"num_bytes":29351651,"num_objects":25,"num_object_clones":2,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":8,"num_read":2577,"num_read_kb":5596,"num_write":644,"num_write_kb":4723,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"39'125","reported_seq":778,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.258690+0000","last_change":"2024-05-08T08:25:06.625296+0000","last_active":"2024-05-08T08:25:30.258690+0000","last_peered":"2024-05-08T08:25:30.258690+0000","last_clean":"2024-05-08T08:25:30.258690+0000","last_became_active":"2024-05-08T08:25:04.605228+0000","last_became_peered":"2024-05-08T08:25:04.605228+0000","last_unstale":"2024-05-08T08:25:30.258690+0000","last_undegraded":"2024-05-08T08:25:30.258690+0000","last_fullsized":"2024-05-08T08:25:30.258690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:26:52.330492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011464940000000001,"stat_sum":{"num_bytes":6545146,"num_objects":21,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":9,"num_read":1060,"num_read_kb":7365,"num_write":112,"num_write_kb":5066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"39'78","reported_seq":754,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.267657+0000","last_change":"2024-05-08T08:25:06.796215+0000","last_active":"2024-05-08T08:25:30.267657+0000","last_peered":"2024-05-08T08:25:30.267657+0000","last_clean":"2024-05-08T08:25:30.267657+0000","last_became_active":"2024-05-08T08:25:04.606180+0000","last_became_peered":"2024-05-08T08:25:04.606180+0000","last_unstale":"2024-05-08T08:25:30.267657+0000","last_undegraded":"2024-05-08T08:25:30.267657+0000","last_fullsized":"2024-05-08T08:25:30.267657+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:42:49.523647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027126400000000001,"stat_sum":{"num_bytes":13918208,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":625,"num_read_kb":11378,"num_write":98,"num_write_kb":1925,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"39'99","reported_seq":429,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.253193+0000","last_change":"2024-05-08T08:25:06.615415+0000","last_active":"2024-05-08T08:25:30.253193+0000","last_peered":"2024-05-08T08:25:30.253193+0000","last_clean":"2024-05-08T08:25:30.253193+0000","last_became_active":"2024-05-08T08:25:04.614760+0000","last_became_peered":"2024-05-08T08:25:04.614760+0000","last_unstale":"2024-05-08T08:25:30.253193+0000","last_undegraded":"2024-05-08T08:25:30.253193+0000","last_fullsized":"2024-05-08T08:25:30.253193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:45:24.369265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034964000000000002,"stat_sum":{"num_bytes":13512170,"num_objects":17,"num_object_clones":5,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":6,"num_read":289,"num_read_kb":4322,"num_write":110,"num_write_kb":4185,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.083739+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:25:30.083739+0000","last_peered":"2024-05-08T08:25:30.083739+0000","last_clean":"2024-05-08T08:25:30.083739+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:25:30.083739+0000","last_undegraded":"2024-05-08T08:25:30.083739+0000","last_fullsized":"2024-05-08T08:25:30.083739+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:32.338505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"39'111","reported_seq":353,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.057356+0000","last_change":"2024-05-08T08:25:06.625323+0000","last_active":"2024-05-08T08:25:30.057356+0000","last_peered":"2024-05-08T08:25:30.057356+0000","last_clean":"2024-05-08T08:25:30.057356+0000","last_became_active":"2024-05-08T08:25:04.606855+0000","last_became_peered":"2024-05-08T08:25:04.606855+0000","last_unstale":"2024-05-08T08:25:30.057356+0000","last_undegraded":"2024-05-08T08:25:30.057356+0000","last_fullsized":"2024-05-08T08:25:30.057356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":111,"log_dups_size":0,"ondisk_log_size":111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:30.739164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011708910000000001,"stat_sum":{"num_bytes":3415398,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":12,"num_read":221,"num_read_kb":1206,"num_write":114,"num_write_kb":2591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.083719+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:25:30.083719+0000","last_peered":"2024-05-08T08:25:30.083719+0000","last_clean":"2024-05-08T08:25:30.083719+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:25:30.083719+0000","last_undegraded":"2024-05-08T08:25:30.083719+0000","last_fullsized":"2024-05-08T08:25:30.083719+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:54.867771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"41'119","reported_seq":863,"reported_epoch":41,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:25:30.440082+0000","last_change":"2024-05-08T08:25:21.002993+0000","last_active":"2024-05-08T08:25:30.440082+0000","last_peered":"2024-05-08T08:25:30.440082+0000","last_clean":"2024-05-08T08:25:18.938588+0000","last_became_active":"2024-05-08T08:25:21.002993+0000","last_became_peered":"2024-05-08T08:25:21.002993+0000","last_unstale":"2024-05-08T08:25:30.440082+0000","last_undegraded":"2024-05-08T08:25:21.002113+0000","last_fullsized":"2024-05-08T08:25:21.002026+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:06:49.541644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847699999999999,"stat_sum":{"num_bytes":33713088,"num_objects":20,"num_object_clones":6,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":10,"num_read":769,"num_read_kb":22553,"num_write":120,"num_write_kb":3034,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":20}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.055324+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:25:30.055324+0000","last_peered":"2024-05-08T08:25:30.055324+0000","last_clean":"2024-05-08T08:25:30.055324+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:25:30.055324+0000","last_undegraded":"2024-05-08T08:25:30.055324+0000","last_fullsized":"2024-05-08T08:25:30.055324+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:43:18.265730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"39'479","reported_seq":1402,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.055475+0000","last_change":"2024-05-08T08:25:06.611208+0000","last_active":"2024-05-08T08:25:30.055475+0000","last_peered":"2024-05-08T08:25:30.055475+0000","last_clean":"2024-05-08T08:25:30.055475+0000","last_became_active":"2024-05-08T08:25:04.608555+0000","last_became_peered":"2024-05-08T08:25:04.608555+0000","last_unstale":"2024-05-08T08:25:30.055475+0000","last_undegraded":"2024-05-08T08:25:30.055475+0000","last_fullsized":"2024-05-08T08:25:30.055475+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:18:19.001382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195331,"stat_sum":{"num_bytes":17456388,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":2966,"num_read_kb":23562,"num_write":729,"num_write_kb":12707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":61,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.055088+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:25:30.055088+0000","last_peered":"2024-05-08T08:25:30.055088+0000","last_clean":"2024-05-08T08:25:30.055088+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:25:30.055088+0000","last_undegraded":"2024-05-08T08:25:30.055088+0000","last_fullsized":"2024-05-08T08:25:30.055088+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"39'123","reported_seq":1101,"reported_epoch":41,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:25:30.080611+0000","last_change":"2024-05-08T08:25:21.001198+0000","last_active":"2024-05-08T08:25:30.080611+0000","last_peered":"2024-05-08T08:25:30.080611+0000","last_clean":"2024-05-08T08:25:20.249043+0000","last_became_active":"2024-05-08T08:25:21.001198+0000","last_became_peered":"2024-05-08T08:25:21.001198+0000","last_unstale":"2024-05-08T08:25:30.080611+0000","last_undegraded":"2024-05-08T08:25:21.000818+0000","last_fullsized":"2024-05-08T08:25:21.000721+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:05.115315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062277100000000005,"stat_sum":{"num_bytes":22861746,"num_objects":20,"num_object_clones":3,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":12,"num_read":1715,"num_read_kb":17856,"num_write":110,"num_write_kb":4454,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":20}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"41'156","reported_seq":956,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.430211+0000","last_change":"2024-05-08T08:25:06.628002+0000","last_active":"2024-05-08T08:25:30.430211+0000","last_peered":"2024-05-08T08:25:30.430211+0000","last_clean":"2024-05-08T08:25:30.430211+0000","last_became_active":"2024-05-08T08:25:04.615077+0000","last_became_peered":"2024-05-08T08:25:04.615077+0000","last_unstale":"2024-05-08T08:25:30.430211+0000","last_undegraded":"2024-05-08T08:25:30.430211+0000","last_fullsized":"2024-05-08T08:25:30.430211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:39:58.113781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057249000000000004,"stat_sum":{"num_bytes":5699937,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":1379,"num_read_kb":7336,"num_write":157,"num_write_kb":5488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":212,"reported_epoch":41,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:25:30.060123+0000","last_change":"2024-05-08T08:25:21.005758+0000","last_active":"2024-05-08T08:25:30.060123+0000","last_peered":"2024-05-08T08:25:30.060123+0000","last_clean":"2024-05-08T08:25:19.722344+0000","last_became_active":"2024-05-08T08:25:21.005758+0000","last_became_peered":"2024-05-08T08:25:21.005758+0000","last_unstale":"2024-05-08T08:25:30.060123+0000","last_undegraded":"2024-05-08T08:25:21.005365+0000","last_fullsized":"2024-05-08T08:25:21.005280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:21:03.389156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":2}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":24,"reported_epoch":41,"state":"active+undersized","last_fresh":"2024-05-08T08:25:30.083699+0000","last_change":"2024-05-08T08:25:21.005812+0000","last_active":"2024-05-08T08:25:30.083699+0000","last_peered":"2024-05-08T08:25:30.083699+0000","last_clean":"2024-05-08T08:24:55.548970+0000","last_became_active":"2024-05-08T08:25:21.005812+0000","last_became_peered":"2024-05-08T08:25:21.005812+0000","last_unstale":"2024-05-08T08:25:30.083699+0000","last_undegraded":"2024-05-08T08:25:30.083699+0000","last_fullsized":"2024-05-08T08:25:21.005149+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:37:46.115602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"40'107","reported_seq":611,"reported_epoch":41,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:25:30.254820+0000","last_change":"2024-05-08T08:25:21.002559+0000","last_active":"2024-05-08T08:25:30.254820+0000","last_peered":"2024-05-08T08:25:30.254820+0000","last_clean":"2024-05-08T08:25:18.936967+0000","last_became_active":"2024-05-08T08:25:21.002559+0000","last_became_peered":"2024-05-08T08:25:21.002559+0000","last_unstale":"2024-05-08T08:25:30.254820+0000","last_undegraded":"2024-05-08T08:25:21.002066+0000","last_fullsized":"2024-05-08T08:25:21.001988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":107,"log_dups_size":0,"ondisk_log_size":107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:18.526364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034192400000000001,"stat_sum":{"num_bytes":15784114,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":475,"num_read_kb":4094,"num_write":110,"num_write_kb":1571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":19}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.060181+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:25:30.060181+0000","last_peered":"2024-05-08T08:25:30.060181+0000","last_clean":"2024-05-08T08:25:30.060181+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:25:30.060181+0000","last_undegraded":"2024-05-08T08:25:30.060181+0000","last_fullsized":"2024-05-08T08:25:30.060181+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"39'114","reported_seq":286,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.254404+0000","last_change":"2024-05-08T08:25:06.615595+0000","last_active":"2024-05-08T08:25:30.254404+0000","last_peered":"2024-05-08T08:25:30.254404+0000","last_clean":"2024-05-08T08:25:30.254404+0000","last_became_active":"2024-05-08T08:25:04.611900+0000","last_became_peered":"2024-05-08T08:25:04.611900+0000","last_unstale":"2024-05-08T08:25:30.254404+0000","last_undegraded":"2024-05-08T08:25:30.254404+0000","last_fullsized":"2024-05-08T08:25:30.254404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":114,"log_dups_size":0,"ondisk_log_size":114,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:54:26.616442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050119499999999996,"stat_sum":{"num_bytes":2437070,"num_objects":15,"num_object_clones":2,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":119,"num_read_kb":1977,"num_write":115,"num_write_kb":518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.057093+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:25:30.057093+0000","last_peered":"2024-05-08T08:25:30.057093+0000","last_clean":"2024-05-08T08:25:30.057093+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:25:30.057093+0000","last_undegraded":"2024-05-08T08:25:30.057093+0000","last_fullsized":"2024-05-08T08:25:30.057093+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"40'93","reported_seq":857,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.255374+0000","last_change":"2024-05-08T08:25:06.610137+0000","last_active":"2024-05-08T08:25:30.255374+0000","last_peered":"2024-05-08T08:25:30.255374+0000","last_clean":"2024-05-08T08:25:30.255374+0000","last_became_active":"2024-05-08T08:25:04.606801+0000","last_became_peered":"2024-05-08T08:25:04.606801+0000","last_unstale":"2024-05-08T08:25:30.255374+0000","last_undegraded":"2024-05-08T08:25:30.255374+0000","last_fullsized":"2024-05-08T08:25:30.255374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:22:46.055654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042592699999999999,"stat_sum":{"num_bytes":56488169,"num_objects":25,"num_object_clones":5,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":15,"num_read":732,"num_read_kb":35870,"num_write":87,"num_write_kb":14463,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.060125+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:25:30.060125+0000","last_peered":"2024-05-08T08:25:30.060125+0000","last_clean":"2024-05-08T08:25:30.060125+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:25:30.060125+0000","last_undegraded":"2024-05-08T08:25:30.060125+0000","last_fullsized":"2024-05-08T08:25:30.060125+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"39'136","reported_seq":1480,"reported_epoch":41,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:25:30.276077+0000","last_change":"2024-05-08T08:25:20.999615+0000","last_active":"2024-05-08T08:25:30.276077+0000","last_peered":"2024-05-08T08:25:30.276077+0000","last_clean":"2024-05-08T08:25:18.986429+0000","last_became_active":"2024-05-08T08:25:20.999615+0000","last_became_peered":"2024-05-08T08:25:20.999615+0000","last_unstale":"2024-05-08T08:25:30.276077+0000","last_undegraded":"2024-05-08T08:25:20.999035+0000","last_fullsized":"2024-05-08T08:25:20.998950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":136,"log_dups_size":0,"ondisk_log_size":136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.120499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001097338,"stat_sum":{"num_bytes":6672535,"num_objects":22,"num_object_clones":4,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":2590,"num_read_kb":5058,"num_write":131,"num_write_kb":5155,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":22}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"41'156","reported_seq":908,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.432830+0000","last_change":"2024-05-08T08:25:06.615379+0000","last_active":"2024-05-08T08:25:30.432830+0000","last_peered":"2024-05-08T08:25:30.432830+0000","last_clean":"2024-05-08T08:25:30.432830+0000","last_became_active":"2024-05-08T08:25:04.901435+0000","last_became_peered":"2024-05-08T08:25:04.901435+0000","last_unstale":"2024-05-08T08:25:30.432830+0000","last_undegraded":"2024-05-08T08:25:30.432830+0000","last_fullsized":"2024-05-08T08:25:30.432830+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:09:24.740270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039550799999999998,"stat_sum":{"num_bytes":4596805,"num_objects":13,"num_object_clones":2,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":2005,"num_read_kb":2144,"num_write":149,"num_write_kb":585,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"41'162","reported_seq":494,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.257569+0000","last_change":"2024-05-08T08:25:06.612336+0000","last_active":"2024-05-08T08:25:30.257569+0000","last_peered":"2024-05-08T08:25:30.257569+0000","last_clean":"2024-05-08T08:25:30.257569+0000","last_became_active":"2024-05-08T08:25:04.615227+0000","last_became_peered":"2024-05-08T08:25:04.615227+0000","last_unstale":"2024-05-08T08:25:30.257569+0000","last_undegraded":"2024-05-08T08:25:30.257569+0000","last_fullsized":"2024-05-08T08:25:30.257569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":162,"log_dups_size":0,"ondisk_log_size":162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:37:22.826658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034968800000000002,"stat_sum":{"num_bytes":21744596,"num_objects":24,"num_object_clones":3,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":10,"num_read":557,"num_read_kb":5148,"num_write":190,"num_write_kb":1995,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"40'121","reported_seq":1012,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.281874+0000","last_change":"2024-05-08T08:25:06.610868+0000","last_active":"2024-05-08T08:25:30.281874+0000","last_peered":"2024-05-08T08:25:30.281874+0000","last_clean":"2024-05-08T08:25:30.281874+0000","last_became_active":"2024-05-08T08:25:04.608456+0000","last_became_peered":"2024-05-08T08:25:04.608456+0000","last_unstale":"2024-05-08T08:25:30.281874+0000","last_undegraded":"2024-05-08T08:25:30.281874+0000","last_fullsized":"2024-05-08T08:25:30.281874+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":121,"log_dups_size":0,"ondisk_log_size":121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:19:09.249574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112149,"stat_sum":{"num_bytes":26940974,"num_objects":28,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":13,"num_read":862,"num_read_kb":3756,"num_write":116,"num_write_kb":3655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"41'128","reported_seq":319,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.203254+0000","last_change":"2024-05-08T08:25:06.624428+0000","last_active":"2024-05-08T08:25:30.203254+0000","last_peered":"2024-05-08T08:25:30.203254+0000","last_clean":"2024-05-08T08:25:30.203254+0000","last_became_active":"2024-05-08T08:25:04.606925+0000","last_became_peered":"2024-05-08T08:25:04.606925+0000","last_unstale":"2024-05-08T08:25:30.203254+0000","last_undegraded":"2024-05-08T08:25:30.203254+0000","last_fullsized":"2024-05-08T08:25:30.203254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:02:59.592078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046376499999999999,"stat_sum":{"num_bytes":46147766,"num_objects":13,"num_object_clones":1,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":8,"num_read":154,"num_read_kb":1885,"num_write":194,"num_write_kb":14606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"39'91","reported_seq":224,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.257452+0000","last_change":"2024-05-08T08:25:06.796419+0000","last_active":"2024-05-08T08:25:30.257452+0000","last_peered":"2024-05-08T08:25:30.257452+0000","last_clean":"2024-05-08T08:25:30.257452+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:25:30.257452+0000","last_undegraded":"2024-05-08T08:25:30.257452+0000","last_fullsized":"2024-05-08T08:25:30.257452+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:09.399625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046517299999999999,"stat_sum":{"num_bytes":1677312,"num_objects":11,"num_object_clones":2,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":6,"num_read":78,"num_read_kb":1694,"num_write":97,"num_write_kb":870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"39'115","reported_seq":284,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.060348+0000","last_change":"2024-05-08T08:25:06.796635+0000","last_active":"2024-05-08T08:25:30.060348+0000","last_peered":"2024-05-08T08:25:30.060348+0000","last_clean":"2024-05-08T08:25:30.060348+0000","last_became_active":"2024-05-08T08:25:04.615004+0000","last_became_peered":"2024-05-08T08:25:04.615004+0000","last_unstale":"2024-05-08T08:25:30.060348+0000","last_undegraded":"2024-05-08T08:25:30.060348+0000","last_fullsized":"2024-05-08T08:25:30.060348+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":115,"log_dups_size":0,"ondisk_log_size":115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.604336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060116800000000001,"stat_sum":{"num_bytes":13369344,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":8,"num_read":123,"num_read_kb":4013,"num_write":114,"num_write_kb":2125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"39'79","reported_seq":493,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.270060+0000","last_change":"2024-05-08T08:25:06.796737+0000","last_active":"2024-05-08T08:25:30.270060+0000","last_peered":"2024-05-08T08:25:30.270060+0000","last_clean":"2024-05-08T08:25:30.270060+0000","last_became_active":"2024-05-08T08:25:04.615942+0000","last_became_peered":"2024-05-08T08:25:04.615942+0000","last_unstale":"2024-05-08T08:25:30.270060+0000","last_undegraded":"2024-05-08T08:25:30.270060+0000","last_fullsized":"2024-05-08T08:25:30.270060+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:40:54.501963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065917600000000003,"stat_sum":{"num_bytes":4083243,"num_objects":16,"num_object_clones":3,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":7,"num_read":368,"num_read_kb":1522,"num_write":74,"num_write_kb":826,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"39'216","reported_seq":621,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.257425+0000","last_change":"2024-05-08T08:25:06.612297+0000","last_active":"2024-05-08T08:25:30.257425+0000","last_peered":"2024-05-08T08:25:30.257425+0000","last_clean":"2024-05-08T08:25:30.257425+0000","last_became_active":"2024-05-08T08:25:04.616212+0000","last_became_peered":"2024-05-08T08:25:04.616212+0000","last_unstale":"2024-05-08T08:25:30.257425+0000","last_undegraded":"2024-05-08T08:25:30.257425+0000","last_fullsized":"2024-05-08T08:25:30.257425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:12.119780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031704699999999999,"stat_sum":{"num_bytes":2588020,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":6,"num_read":933,"num_read_kb":3050,"num_write":302,"num_write_kb":1686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"41'144","reported_seq":794,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.257495+0000","last_change":"2024-05-08T08:25:06.796231+0000","last_active":"2024-05-08T08:25:30.257495+0000","last_peered":"2024-05-08T08:25:30.257495+0000","last_clean":"2024-05-08T08:25:30.257495+0000","last_became_active":"2024-05-08T08:25:04.615031+0000","last_became_peered":"2024-05-08T08:25:04.615031+0000","last_unstale":"2024-05-08T08:25:30.257495+0000","last_undegraded":"2024-05-08T08:25:30.257495+0000","last_fullsized":"2024-05-08T08:25:30.257495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":144,"log_dups_size":0,"ondisk_log_size":144,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:47:12.902464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040064000000000001,"stat_sum":{"num_bytes":19898707,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":777,"num_read_kb":16036,"num_write":165,"num_write_kb":2898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"39'100","reported_seq":892,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.271623+0000","last_change":"2024-05-08T08:25:06.628036+0000","last_active":"2024-05-08T08:25:30.271623+0000","last_peered":"2024-05-08T08:25:30.271623+0000","last_clean":"2024-05-08T08:25:30.271623+0000","last_became_active":"2024-05-08T08:25:04.609651+0000","last_became_peered":"2024-05-08T08:25:04.609651+0000","last_unstale":"2024-05-08T08:25:30.271623+0000","last_undegraded":"2024-05-08T08:25:30.271623+0000","last_fullsized":"2024-05-08T08:25:30.271623+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":100,"log_dups_size":0,"ondisk_log_size":100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:15:56.323277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050895200000000002,"stat_sum":{"num_bytes":13969153,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":8,"num_read":748,"num_read_kb":13997,"num_write":94,"num_write_kb":9205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"40'438","reported_seq":1130,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-05-08T08:25:30.269390+0000","last_change":"2024-05-08T08:25:06.796497+0000","last_active":"2024-05-08T08:25:30.269390+0000","last_peered":"2024-05-08T08:25:30.269390+0000","last_clean":"2024-05-08T08:25:30.269390+0000","last_became_active":"2024-05-08T08:25:04.616040+0000","last_became_peered":"2024-05-08T08:25:04.616040+0000","last_unstale":"2024-05-08T08:25:30.269390+0000","last_undegraded":"2024-05-08T08:25:30.269390+0000","last_fullsized":"2024-05-08T08:25:30.269390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":438,"log_dups_size":0,"ondisk_log_size":438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:18:33.047377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050921500000000004,"stat_sum":{"num_bytes":36578733,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":2614,"num_read_kb":9279,"num_write":732,"num_write_kb":4483,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":580177183,"num_objects":606,"num_object_clones":112,"num_object_copies":1212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":119,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":606,"num_whiteouts":289,"num_read":35677,"num_read_kb":270540,"num_write":7116,"num_write_kb":148279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21278720,"data_stored":206348033,"data_compressed":5626431,"data_compressed_allocated":17567744,"data_compressed_original":203996169,"omap_allocated":0,"internal_metadata":0},"log_size":5569,"ondisk_log_size":5569,"up":58,"acting":58,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":1,"acting":1,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":15,"acting":15,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39048,"kb_used_data":3708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332792,"statfs":{"total":96636764160,"available":96596779008,"internally_reserved":0,"allocated":3796992,"data_stored":37050617,"data_compressed":941435,"data_compressed_allocated":2772992,"data_compressed_original":36472048,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1388,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1421312,"data_stored":8926379,"data_compressed":219644,"data_compressed_allocated":753664,"data_compressed_original":8605696,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34920,"kb_used_data":3736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336920,"statfs":{"total":96636764160,"available":96601006080,"internally_reserved":0,"allocated":3825664,"data_stored":35629426,"data_compressed":819059,"data_compressed_allocated":2891776,"data_compressed_original":35118111,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39252,"kb_used_data":3916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94332588,"statfs":{"total":96636764160,"available":96596570112,"internally_reserved":0,"allocated":4009984,"data_stored":30911558,"data_compressed":1015413,"data_compressed_allocated":2830336,"data_compressed_original":30252094,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40508,"kb_used_data":5100,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94331332,"statfs":{"total":96636764160,"available":96595283968,"internally_reserved":0,"allocated":5222400,"data_stored":51051074,"data_compressed":1135311,"data_compressed_allocated":4177920,"data_compressed_original":50502081,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339680,"statfs":{"total":96636764160,"available":96603832320,"internally_reserved":0,"allocated":999424,"data_stored":3678244,"data_compressed":151784,"data_compressed_allocated":360448,"data_compressed_original":3350528,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574855,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32036,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339804,"statfs":{"total":96636764160,"available":96603959296,"internally_reserved":0,"allocated":880640,"data_stored":3658022,"data_compressed":160912,"data_compressed_allocated":360448,"data_compressed_original":3395584,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38616,"kb_used_data":3276,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333224,"statfs":{"total":96636764160,"available":96597221376,"internally_reserved":0,"allocated":3354624,"data_stored":28232445,"data_compressed":778664,"data_compressed_allocated":2314240,"data_compressed_original":27676982,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2932736,"data_stored":28068535,"data_compressed":794021,"data_compressed_allocated":2330624,"data_compressed_original":27701558,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":466944,"data_stored":2873622,"data_compressed":109909,"data_compressed_allocated":274432,"data_compressed_original":2748416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":536576,"data_stored":3479900,"data_compressed":151784,"data_compressed_allocated":360448,"data_compressed_original":3350528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4755456,"data_stored":49502535,"data_compressed":1170404,"data_compressed_allocated":4128768,"data_compressed_original":49124296,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3629056,"data_stored":30783559,"data_compressed":1048648,"data_compressed_allocated":2871296,"data_compressed_original":30303084,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4710400,"data_stored":46643799,"data_compressed":1195529,"data_compressed_allocated":4112384,"data_compressed_original":46263983,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":958464,"data_stored":8728035,"data_compressed":219644,"data_compressed_allocated":753664,"data_compressed_original":8605696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3289088,"data_stored":36268048,"data_compressed":936492,"data_compressed_allocated":2736128,"data_compressed_original":35898608,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:31.706 INFO:tasks.thrashosds.thrasher:Setting 3.8 to [5, 0, 6, 2] 2024-05-08T08:25:31.706 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '3.8', '5', '0', '6', '2'] 2024-05-08T08:25:31.707 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap-items 3.8 5 0 6 2 2024-05-08T08:25:31.745 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:31.753+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:31.793 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:31.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:31.846 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:31.854+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:31.914 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:31.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:31.941 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:31.948+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:32.042 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:32.049+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:32.080 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:32.080 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:32.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:32.142 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:32.150+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:32.198 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-05-08T08:25:32.208 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:32.208 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:32.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:32.243 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:32.250+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:32.344 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:32.351+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:32.377 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:32.378 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:32.387 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:32.387 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.403 INFO:teuthology.orchestra.run.smithi031.stdout:2481 write 0xdf7ea46 thru 0xdf82fe4 (0x459f bytes) 2024-05-08T08:25:32.405 INFO:teuthology.orchestra.run.smithi031.stdout:2483 write 0x1b0b537 thru 0x1b18ffd (0xdac7 bytes) 2024-05-08T08:25:32.409 INFO:teuthology.orchestra.run.smithi031.stdout:2484 punch from 0x7487665 to 0x748b059, (0x39f4 bytes) 2024-05-08T08:25:32.409 INFO:teuthology.orchestra.run.smithi031.stdout:2485 read 0x9ef4d3c thru 0x9ef59ec (0xcb1 bytes) 2024-05-08T08:25:32.409 INFO:teuthology.orchestra.run.smithi031.stdout:2487 read 0xbe7435e thru 0xbe81267 (0xcf0a bytes) 2024-05-08T08:25:32.409 INFO:teuthology.orchestra.run.smithi031.stdout:2488 write 0xa8e793d thru 0xa8e86c9 (0xd8d bytes) 2024-05-08T08:25:32.412 INFO:teuthology.orchestra.run.smithi031.stdout:2489 writesame 0x7a41c63 thru 0x7a4a294 data_size 0x59(0x8632 bytes) 2024-05-08T08:25:32.415 INFO:teuthology.orchestra.run.smithi031.stdout:2490 write 0xf5b31f thru 0xf60f4e (0x5c30 bytes) 2024-05-08T08:25:32.425 INFO:teuthology.orchestra.run.smithi031.stdout:2491 write 0xe5db498 thru 0xe5ea9ba (0xf523 bytes) 2024-05-08T08:25:32.428 INFO:teuthology.orchestra.run.smithi031.stdout:2493 punch from 0xc6717f6 to 0xc67cb94, (0xb39e bytes) 2024-05-08T08:25:32.436 INFO:teuthology.orchestra.run.smithi031.stdout:2494 trunc from 0xe5ea9bb to 0x7824a74 2024-05-08T08:25:32.445 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:32.451+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:32.450 INFO:teuthology.orchestra.run.smithi031.stdout:2495 write 0x4fe5161 thru 0x4fe9ace (0x496e bytes) 2024-05-08T08:25:32.455 INFO:teuthology.orchestra.run.smithi031.stdout:2496 punch from 0x1636040 to 0x1643982, (0xd942 bytes) 2024-05-08T08:25:32.455 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.455 INFO:teuthology.orchestra.run.smithi031.stdout:2499 write 0x2c48093 thru 0x2c4e61b (0x6589 bytes) 2024-05-08T08:25:32.458 INFO:teuthology.orchestra.run.smithi031.stdout:2500 read 0x283e1f3 thru 0x283f6bd (0x14cb bytes) 2024-05-08T08:25:32.459 INFO:teuthology.orchestra.run.smithi031.stdout:2501 write 0x56b5a8e thru 0x56c513f (0xf6b2 bytes) 2024-05-08T08:25:32.463 INFO:teuthology.orchestra.run.smithi031.stdout:2502 punch from 0x6cc8abc to 0x6ccb6f5, (0x2c39 bytes) 2024-05-08T08:25:32.463 INFO:teuthology.orchestra.run.smithi031.stdout:2503 read 0x2f7f9c3 thru 0x2f82262 (0x28a0 bytes) 2024-05-08T08:25:32.463 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.470 INFO:teuthology.orchestra.run.smithi031.stdout:2506 trunc from 0x7824a74 to 0xda2e62d 2024-05-08T08:25:32.475 INFO:teuthology.orchestra.run.smithi031.stdout:2507 write 0xd914608 thru 0xd91e60a (0xa003 bytes) 2024-05-08T08:25:32.478 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.478 INFO:teuthology.orchestra.run.smithi031.stdout:2509 read 0x20c619d thru 0x20d111b (0xaf7f bytes) 2024-05-08T08:25:32.490 INFO:teuthology.orchestra.run.smithi031.stdout:2510 trunc from 0xda2e62d to 0x2cac111 2024-05-08T08:25:32.500 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:32.500 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:32.506 INFO:teuthology.orchestra.run.smithi031.stdout:2511 writesame 0xaba203 thru 0xac0eba data_size 0x62(0x6cb8 bytes) 2024-05-08T08:25:32.510 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:32.517 INFO:teuthology.orchestra.run.smithi031.stdout:2512 trunc from 0x2cac111 to 0x8da07ca 2024-05-08T08:25:32.523 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.524 INFO:teuthology.orchestra.run.smithi031.stdout:2515 trunc from 0x8da07ca to 0x7f82c9b 2024-05-08T08:25:32.530 INFO:teuthology.orchestra.run.smithi031.stdout:2516 read 0x37e4338 thru 0x37eb3c3 (0x708c bytes) 2024-05-08T08:25:32.530 INFO:teuthology.orchestra.run.smithi031.stdout:2517 write 0x127a7dc thru 0x1288564 (0xdd89 bytes) 2024-05-08T08:25:32.534 INFO:teuthology.orchestra.run.smithi031.stdout:2518 read 0x5b6bee0 thru 0x5b7a2dd (0xe3fe bytes) 2024-05-08T08:25:32.541 INFO:teuthology.orchestra.run.smithi031.stdout:2520 trunc from 0x7f82c9b to 0x15114d0 2024-05-08T08:25:32.550 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:32.557+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:32.559 INFO:teuthology.orchestra.run.smithi031.stdout:2521 read 0x140f490 thru 0x141e6d1 (0xf242 bytes) 2024-05-08T08:25:32.560 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.560 INFO:teuthology.orchestra.run.smithi031.stdout:2523 read 0x26aca3 thru 0x26ad41 (0x9f bytes) 2024-05-08T08:25:32.569 INFO:teuthology.orchestra.run.smithi031.stdout:2524 write 0x3dba13e thru 0x3dbb0bc (0xf7f bytes) 2024-05-08T08:25:32.571 INFO:teuthology.orchestra.run.smithi031.stdout:2526 writesame 0x2adae17 thru 0x2ae2a8e data_size 0x1c(0x7c78 bytes) 2024-05-08T08:25:32.577 INFO:teuthology.orchestra.run.smithi031.stdout:2527 punch from 0x3323346 to 0x3333036, (0xfcf0 bytes) 2024-05-08T08:25:32.577 INFO:teuthology.orchestra.run.smithi031.stdout:2528 read 0x150ef32 thru 0x150f05c (0x12b bytes) 2024-05-08T08:25:32.592 INFO:teuthology.orchestra.run.smithi031.stdout:2529 write 0xb97179b thru 0xb97312b (0x1991 bytes) 2024-05-08T08:25:32.595 INFO:teuthology.orchestra.run.smithi031.stdout:2530 write 0x5c1e3ee thru 0x5c2165f (0x3272 bytes) 2024-05-08T08:25:32.599 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.599 INFO:teuthology.orchestra.run.smithi031.stdout:2532 writesame 0x8b511e2 thru 0x8b57674 data_size 0x1(0x6493 bytes) 2024-05-08T08:25:32.613 INFO:teuthology.orchestra.run.smithi031.stdout:2533 read 0x28d781d thru 0x28d81e1 (0x9c5 bytes) 2024-05-08T08:25:32.613 INFO:teuthology.orchestra.run.smithi031.stdout:2535 write 0x36d43a9 thru 0x36dd312 (0x8f6a bytes) 2024-05-08T08:25:32.617 INFO:teuthology.orchestra.run.smithi031.stdout:2536 read 0x4fbf5e5 thru 0x4fc6a01 (0x741d bytes) 2024-05-08T08:25:32.617 INFO:teuthology.orchestra.run.smithi031.stdout:2537 punch from 0x3862a7e to 0x3863752, (0xcd4 bytes) 2024-05-08T08:25:32.617 INFO:teuthology.orchestra.run.smithi031.stdout:2538 read 0x4ec7415 thru 0x4ed209e (0xac8a bytes) 2024-05-08T08:25:32.624 INFO:teuthology.orchestra.run.smithi031.stdout:2539 trunc from 0xb97312c to 0x5819755 2024-05-08T08:25:32.635 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.644 INFO:teuthology.orchestra.run.smithi031.stdout:2541 write 0x8951373 thru 0x895d19f (0xbe2d bytes) 2024-05-08T08:25:32.646 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:32.652+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:32.648 INFO:teuthology.orchestra.run.smithi031.stdout:2542 punch from 0x7fc6816 to 0x7fd1bb2, (0xb39c bytes) 2024-05-08T08:25:32.648 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.648 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.659 INFO:teuthology.orchestra.run.smithi031.stdout:2546 write 0xd9692d2 thru 0xd969512 (0x241 bytes) 2024-05-08T08:25:32.661 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:32.662 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:32.662 INFO:teuthology.orchestra.run.smithi031.stdout:2548 write 0x3cae3b6 thru 0x3cb1da4 (0x39ef bytes) 2024-05-08T08:25:32.665 INFO:teuthology.orchestra.run.smithi031.stdout:2551 write 0x7e0850e thru 0x7e15e79 (0xd96c bytes) 2024-05-08T08:25:32.668 INFO:teuthology.orchestra.run.smithi031.stdout:2552 read 0x2692fa8 thru 0x26a00e5 (0xd13e bytes) 2024-05-08T08:25:32.668 INFO:teuthology.orchestra.run.smithi031.stdout:2553 read 0x4e786c9 thru 0x4e85e58 (0xd790 bytes) 2024-05-08T08:25:32.671 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:32.673 INFO:teuthology.orchestra.run.smithi031.stdout:2554 trunc from 0xd969513 to 0x956ff8a 2024-05-08T08:25:32.685 INFO:teuthology.orchestra.run.smithi031.stdout:2555 write 0x49ec98 thru 0x4a259d (0x3906 bytes) 2024-05-08T08:25:32.696 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.696 INFO:teuthology.orchestra.run.smithi031.stdout:2558 read 0x4c25033 thru 0x4c2fd54 (0xad22 bytes) 2024-05-08T08:25:32.700 INFO:teuthology.orchestra.run.smithi031.stdout:2560 trunc from 0x956ff8a to 0x6562bbe 2024-05-08T08:25:32.722 INFO:teuthology.orchestra.run.smithi031.stdout:2561 writesame 0xcb5040a thru 0xcb50d42 data_size 0x1(0x939 bytes) 2024-05-08T08:25:32.730 INFO:teuthology.orchestra.run.smithi031.stdout:2562 trunc from 0xcb50d43 to 0x5a962c6 2024-05-08T08:25:32.735 INFO:teuthology.orchestra.run.smithi031.stdout:2563 read 0x1a5db0 thru 0x1aea51 (0x8ca2 bytes) 2024-05-08T08:25:32.736 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.736 INFO:teuthology.orchestra.run.smithi031.stdout:2566 read 0x40eba23 thru 0x40f06a6 (0x4c84 bytes) 2024-05-08T08:25:32.737 INFO:teuthology.orchestra.run.smithi031.stdout:2568 read 0x4d27c91 thru 0x4d33e74 (0xc1e4 bytes) 2024-05-08T08:25:32.737 INFO:teuthology.orchestra.run.smithi031.stdout:2569 write 0xf07380 thru 0xf0f1d6 (0x7e57 bytes) 2024-05-08T08:25:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:2570 punch from 0x26b80f2 to 0x26c66ec, (0xe5fa bytes) 2024-05-08T08:25:32.739 INFO:teuthology.orchestra.run.smithi031.stdout:2571 write 0x241ad23 thru 0x24279d9 (0xccb7 bytes) 2024-05-08T08:25:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:2572 read 0x58f99f7 thru 0x5907331 (0xd93b bytes) 2024-05-08T08:25:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:2574 read 0x439bf0d thru 0x439f3ed (0x34e1 bytes) 2024-05-08T08:25:32.742 INFO:teuthology.orchestra.run.smithi031.stdout:2575 writesame 0x17e5feb thru 0x17e6902 data_size 0xc(0x918 bytes) 2024-05-08T08:25:32.743 INFO:teuthology.orchestra.run.smithi031.stdout:2577 read 0x4587ccc thru 0x458de2f (0x6164 bytes) 2024-05-08T08:25:32.744 INFO:teuthology.orchestra.run.smithi031.stdout:2580 writesame 0x2cad20d thru 0x2cb36f6 data_size 0x1(0x64ea bytes) 2024-05-08T08:25:32.747 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:32.754+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:32.757 INFO:teuthology.orchestra.run.smithi031.stdout:2581 punch from 0x1b200f5 to 0x1b2b72f, (0xb63a bytes) 2024-05-08T08:25:32.757 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.771 INFO:teuthology.orchestra.run.smithi031.stdout:2583 writesame 0xc8f96bf thru 0xc8fb46d data_size 0x11(0x1daf bytes) 2024-05-08T08:25:32.771 INFO:teuthology.orchestra.run.smithi031.stdout:2584 read 0x34aebbe thru 0x34b80d3 (0x9516 bytes) 2024-05-08T08:25:32.771 INFO:teuthology.orchestra.run.smithi031.stdout:2587 writesame 0x26bb4ad thru 0x26c5ed0 data_size 0x2(0xaa24 bytes) 2024-05-08T08:25:32.779 INFO:teuthology.orchestra.run.smithi031.stdout:2589 read 0xbb60975 thru 0xbb6d086 (0xc712 bytes) 2024-05-08T08:25:32.779 INFO:teuthology.orchestra.run.smithi031.stdout:2590 writesame 0x2f19cd2 thru 0x2f223de data_size 0xb(0x870d bytes) 2024-05-08T08:25:32.783 INFO:teuthology.orchestra.run.smithi031.stdout:2591 writesame 0x89d93e2 thru 0x89e0a4f data_size 0x6(0x766e bytes) 2024-05-08T08:25:32.788 INFO:teuthology.orchestra.run.smithi031.stdout:2592 write 0x3bb9514 thru 0x3bbdccf (0x47bc bytes) 2024-05-08T08:25:32.791 INFO:teuthology.orchestra.run.smithi031.stdout:2593 write 0x31bf9c4 thru 0x31cc5ec (0xcc29 bytes) 2024-05-08T08:25:32.796 INFO:teuthology.orchestra.run.smithi031.stdout:2595 trunc from 0xc8fb46e to 0xd85828b 2024-05-08T08:25:32.801 INFO:teuthology.orchestra.run.smithi031.stdout:2596 punch from 0x37eef10 to 0x37ef2b9, (0x3a9 bytes) 2024-05-08T08:25:32.801 INFO:teuthology.orchestra.run.smithi031.stdout:2597 writesame 0x7d6b746 thru 0x7d716fa data_size 0x1(0x5fb5 bytes) 2024-05-08T08:25:32.821 INFO:teuthology.orchestra.run.smithi031.stdout:2598 trunc from 0xd85828b to 0x8567893 2024-05-08T08:25:32.825 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:32.826 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:32.831 INFO:teuthology.orchestra.run.smithi031.stdout:2599 read 0x5cdcd4c thru 0x5cdfcbb (0x2f70 bytes) 2024-05-08T08:25:32.831 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.831 INFO:teuthology.orchestra.run.smithi031.stdout:2602 punch from 0x20eb651 to 0x20f486b, (0x921a bytes) 2024-05-08T08:25:32.831 INFO:teuthology.orchestra.run.smithi031.stdout:2603 read 0x5626293 thru 0x562651a (0x288 bytes) 2024-05-08T08:25:32.831 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:32.831 INFO:teuthology.orchestra.run.smithi031.stdout:2607 punch from 0xd19fc4 to 0xd24679, (0xa6b5 bytes) 2024-05-08T08:25:32.836 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:32.842 INFO:teuthology.orchestra.run.smithi031.stdout:2608 write 0xd1f3e5f thru 0xd1f4303 (0x4a5 bytes) 2024-05-08T08:25:32.856 INFO:teuthology.orchestra.run.smithi031.stdout:2611 trunc from 0xd1f4304 to 0x3184f60 2024-05-08T08:25:32.871 INFO:teuthology.orchestra.run.smithi031.stdout:2614 trunc from 0x3184f60 to 0x85b109 2024-05-08T08:25:32.880 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:32.887+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:32.902 INFO:teuthology.orchestra.run.smithi031.stdout:2616 writesame 0x64ecd79 thru 0x64ef3be data_size 0x2(0x2646 bytes) 2024-05-08T08:25:32.914 INFO:teuthology.orchestra.run.smithi031.stdout:2619 write 0xc279393 thru 0xc28039e (0x700c bytes) 2024-05-08T08:25:32.917 INFO:teuthology.orchestra.run.smithi031.stdout:2620 write 0x54e1f5e thru 0x54e6f5b (0x4ffe bytes) 2024-05-08T08:25:32.933 INFO:teuthology.orchestra.run.smithi031.stdout:2621 trunc from 0xc28039f to 0x8ca5dc 2024-05-08T08:25:32.949 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:32.956+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:32.950 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:32.950 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:32.953 INFO:teuthology.orchestra.run.smithi031.stdout:2622 write 0x3fe4c61 thru 0x3feeb43 (0x9ee3 bytes) 2024-05-08T08:25:32.957 INFO:teuthology.orchestra.run.smithi031.stdout:2623 read 0x12d535c thru 0x12d72bc (0x1f61 bytes) 2024-05-08T08:25:32.957 INFO:teuthology.orchestra.run.smithi031.stdout:2624 read 0xb463eb thru 0xb4c0cc (0x5ce2 bytes) 2024-05-08T08:25:32.957 INFO:teuthology.orchestra.run.smithi031.stdout:2626 read 0xae1ce4 thru 0xae65b4 (0x48d1 bytes) 2024-05-08T08:25:32.958 INFO:teuthology.orchestra.run.smithi031.stdout:2627 read 0x249ce9c thru 0x24a320f (0x6374 bytes) 2024-05-08T08:25:32.958 INFO:teuthology.orchestra.run.smithi031.stdout:2629 write 0x3f416e8 thru 0x3f423e2 (0xcfb bytes) 2024-05-08T08:25:32.959 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:32.960 INFO:teuthology.orchestra.run.smithi031.stdout:2630 write 0x8ff9a9 thru 0x90ded5 (0xe52d bytes) 2024-05-08T08:25:32.973 INFO:teuthology.orchestra.run.smithi031.stdout:2631 trunc from 0x3feeb44 to 0xe3304e1 2024-05-08T08:25:32.980 INFO:teuthology.orchestra.run.smithi031.stdout:2632 writesame 0x933643e thru 0x9345b46 data_size 0x1(0xf709 bytes) 2024-05-08T08:25:33.010 INFO:teuthology.orchestra.run.smithi031.stdout:2634 write 0xdf081cc thru 0xdf14b9c (0xc9d1 bytes) 2024-05-08T08:25:33.020 INFO:teuthology.orchestra.run.smithi031.stdout:2636 trunc from 0xe3304e1 to 0x7c75f65 2024-05-08T08:25:33.030 INFO:teuthology.orchestra.run.smithi031.stdout:2637 read 0x3daac5f thru 0x3db6461 (0xb803 bytes) 2024-05-08T08:25:33.030 INFO:teuthology.orchestra.run.smithi031.stdout:2638 writesame 0x3ce4240 thru 0x3cf1a8f data_size 0x8(0xd850 bytes) 2024-05-08T08:25:33.034 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:33.034 INFO:teuthology.orchestra.run.smithi031.stdout:2640 writesame 0x2999067 thru 0x29a67c5 data_size 0x1(0xd75f bytes) 2024-05-08T08:25:33.050 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:33.058+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:33.052 INFO:teuthology.orchestra.run.smithi031.stdout:2642 read 0x800cde thru 0x81053e (0xf861 bytes) 2024-05-08T08:25:33.053 INFO:teuthology.orchestra.run.smithi031.stdout:2645 punch from 0x42423de to 0x4242eb3, (0xad5 bytes) 2024-05-08T08:25:33.053 INFO:teuthology.orchestra.run.smithi031.stdout:2647 write 0x56332d thru 0x566259 (0x2f2d bytes) 2024-05-08T08:25:33.060 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.8 pg_upmap_items mapping to [5->0,6->2] 2024-05-08T08:25:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:2648 read 0x5cc6272 thru 0x5cc859f (0x232e bytes) 2024-05-08T08:25:33.066 INFO:teuthology.orchestra.run.smithi031.stdout:2649 punch from 0x78d5311 to 0x78e2518, (0xd207 bytes) 2024-05-08T08:25:33.069 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:25:33.076 INFO:teuthology.orchestra.run.smithi031.stdout:2650 writesame 0x807281c thru 0x8075036 data_size 0x1(0x281b bytes) 2024-05-08T08:25:33.088 INFO:teuthology.orchestra.run.smithi031.stdout:2651 write 0xd6ffa32 thru 0xd70426f (0x483e bytes) 2024-05-08T08:25:33.091 INFO:teuthology.orchestra.run.smithi031.stdout:2652 read 0x6bd2117 thru 0x6bde18b (0xc075 bytes) 2024-05-08T08:25:33.091 INFO:teuthology.orchestra.run.smithi031.stdout:2653 clone 15 order 19 su 131072 sc 7 2024-05-08T08:25:33.110 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:33.110 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:33.119 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:33.151 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:33.159+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:33.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:33.224 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:33.224 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:33.233 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:33.277 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:33.284+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:33.307 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:33.307 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:33.307 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:33.325 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:33.325 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:33.340 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:33.340 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:33.342 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:33.342 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:33.349 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:33.360 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:33.360 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:33.377 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:33.377 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:33.390 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:33.397+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:33.394 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:33.395 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:33.412 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:33.415 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:33.416 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:33.453 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:33.460+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:33.495 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:33.496 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:33.496 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:33.496 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:33.496 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:33.504 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:33.554 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:33.562+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:33.611 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1739 3.a 3:516cf025:::rbd_data.119116cc7dd0.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,write 90112~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.178958+0000", 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.438975697, 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039215049999999996, 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1739 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178958+0000", 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178958+0000", 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178960+0000", 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4589999999999999e-06 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178973+0000", 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.364e-05 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178986+0000", 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2678e-05 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.178993+0000", 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2269999999999996e-06 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179041+0000", 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7710999999999998e-05 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179138+0000", 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6791000000000002e-05 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179260+0000", 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000122087 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.179955+0000", 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00069502699999999997 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.182776+0000", 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028215900000000001 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.182838+0000", 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1700999999999998e-05 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.182880+0000", 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1594000000000002e-05 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1815 3.10 3:09da06bb:::rbd_data.119116cc7dd0.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.376734004999999, 2024-05-08T08:25:33.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057340569999999999, 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1815 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241202+0000", 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241211+0000", 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6940000000000001e-06 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241228+0000", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7487e-05 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241239+0000", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0954999999999999e-05 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241305+0000", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5696000000000003e-05 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241432+0000", 2024-05-08T08:25:33.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000127639 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.242178+0000", 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074582499999999998 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.245608+0000", 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003430191 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246869+0000", 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001261096 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246904+0000", 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5024999999999998e-05 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246934+0000", 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9519999999999999e-05 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1817 3.a 3:525853b5:::rbd_data.119116cc7dd0.0000000000000148:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:33.622 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.244093+0000", 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.373841011, 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003967043, 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1817 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244093+0000", 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244093+0000", 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244094+0000", 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0559999999999999e-06 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244099+0000", 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8969999999999999e-06 2024-05-08T08:25:33.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244109+0000", 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0394000000000001e-05 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244114+0000", 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2009999999999998e-06 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244167+0000", 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2867000000000001e-05 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244305+0000", 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000137494 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.244964+0000", 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000659601 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.247758+0000", 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027938189999999999 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.248002+0000", 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000243584 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.248031+0000", 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8906000000000001e-05 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.248060+0000", 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9224e-05 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.627 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1904 3.10 3:0d6c878d:::rbd_data.119116cc7dd0.0000000000000377:head [set-alloc-hint object_size 262144 write_size 262144,write 110592~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.328225+0000", 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.289708660999999, 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025701019999999999, 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1904 2024-05-08T08:25:33.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328225+0000", 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328225+0000", 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328227+0000", 2024-05-08T08:25:33.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3179999999999999e-06 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328236+0000", 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9059999999999997e-06 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328248+0000", 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1883e-05 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328254+0000", 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8830000000000003e-06 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328308+0000", 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3431999999999999e-05 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328398+0000", 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0149999999999996e-05 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.328481+0000", 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2947000000000004e-05 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.329965+0000", 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001483881 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330734+0000", 2024-05-08T08:25:33.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00076957600000000001 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330766+0000", 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2206000000000003e-05 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.330795+0000", 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8920000000000001e-05 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1944 3.10 3:0d6c878d:::rbd_data.119116cc7dd0.0000000000000377:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.371574+0000", 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.246359658999999, 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037156670000000002, 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1944 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371574+0000", 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371574+0000", 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371576+0000", 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4750000000000001e-06 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371582+0000", 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6109999999999999e-06 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371599+0000", 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6833999999999998e-05 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371607+0000", 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2369999999999992e-06 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371672+0000", 2024-05-08T08:25:33.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4795999999999994e-05 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371764+0000", 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1527999999999995e-05 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.371865+0000", 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000101094 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.372589+0000", 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00072437300000000005 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.375181+0000", 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025916009999999998 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.375243+0000", 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1867999999999998e-05 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.375290+0000", 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7250000000000003e-05 2024-05-08T08:25:33.638 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1990 3.10 3:09da06bb:::rbd_data.119116cc7dd0.0000000000000146:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.412463+0000", 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.205471403000001, 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025719789999999998, 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1990 2024-05-08T08:25:33.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412463+0000", 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412463+0000", 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412464+0000", 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4100000000000001e-06 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412469+0000", 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7480000000000001e-06 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412481+0000", 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1951e-05 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412486+0000", 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:25:33.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412538+0000", 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1539000000000002e-05 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412593+0000", 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5114999999999997e-05 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.412667+0000", 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4078000000000005e-05 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.413251+0000", 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058439400000000002 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414941+0000", 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016893259999999999 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414988+0000", 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7324000000000003e-05 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415034+0000", 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6554000000000002e-05 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1992 3.a 3:525853b5:::rbd_data.119116cc7dd0.0000000000000148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.202995377000001, 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005824053, 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.644 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1992 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414940+0000", 2024-05-08T08:25:33.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.029e-06 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414946+0000", 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2890000000000003e-06 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414957+0000", 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1280999999999999e-05 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414966+0000", 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:25:33.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415026+0000", 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0624000000000001e-05 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415093+0000", 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6752000000000001e-05 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415154+0000", 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1016e-05 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417565+0000", 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024105730000000001 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420666+0000", 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031012219999999998 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420721+0000", 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5260000000000003e-05 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420763+0000", 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1519e-05 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:1999 3.1e e23/18 3:7e7612a5:::rbd_data.119116cc7dd0.0000000000000179:head v 23'16, mlcod=23'14)", 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.416681+0000", 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.201252712999999, 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040606110000000004, 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.649 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416681+0000", 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416681+0000", 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416682+0000", 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1299999999999999e-07 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416687+0000", 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9969999999999998e-06 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416696+0000", 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0620000000000007e-06 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416702+0000", 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5670000000000002e-06 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416742+0000", 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0893000000000003e-05 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.416757+0000", 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4686e-05 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420701+0000", 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039436419999999998 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420742+0000", 2024-05-08T08:25:33.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1050999999999997e-05 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2312 3.a 3:535385f0:::rbd_children:head [call rbd.add_child in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.692314+0000", 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 21.925619987000001, 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002844868, 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.653 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2312 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692314+0000", 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692314+0000", 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692316+0000", 2024-05-08T08:25:33.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5600000000000001e-06 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692329+0000", 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2933e-05 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692363+0000", 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3213000000000002e-05 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692378+0000", 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5753e-05 2024-05-08T08:25:33.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692510+0000", 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013140100000000001 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.692521+0000", 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1143e-05 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694159+0000", 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016379910000000001 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694222+0000", 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2932e-05 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694337+0000", 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00011511899999999999 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.694869+0000", 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053169100000000004 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.695100+0000", 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000231177 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.695133+0000", 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3526999999999999e-05 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.695159+0000", 2024-05-08T08:25:33.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5428000000000001e-05 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2330 3.a 3:516cf025:::rbd_data.119116cc7dd0.0000000000000013:head [stat,delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.707370+0000", 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 21.910564386000001, 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042311270000000003, 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2330 2024-05-08T08:25:33.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707370+0000", 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707370+0000", 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707371+0000", 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0589999999999999e-06 2024-05-08T08:25:33.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707376+0000", 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-06 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707392+0000", 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6047000000000001e-05 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707400+0000", 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.106e-06 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707476+0000", 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6145999999999999e-05 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707556+0000", 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9507999999999998e-05 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707750+0000", 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019481899999999999 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708939+0000", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001188789 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711503+0000", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002563835 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711558+0000", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4619e-05 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711601+0000", 2024-05-08T08:25:33.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2959e-05 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:2334 3.1e e24/18 3:7c0c4f5a:::rbd_data.119116cc7dd0.0000000000000017:head v 24'23, mlcod=23'20)", 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.707980+0000", 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 21.909953591000001, 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040302740000000004, 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707980+0000", 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707980+0000", 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707981+0000", 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.707990+0000", 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0289999999999994e-06 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708029+0000", 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9496999999999998e-05 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708037+0000", 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1130000000000001e-06 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708122+0000", 2024-05-08T08:25:33.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4745000000000003e-05 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.708151+0000", 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8765e-05 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.711963+0000", 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038124299999999999 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.712011+0000", 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299999999999998e-05 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.667 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:6773 3.1e e28/18 3:7afff813:::rbd_data.119152d8614e.0000000000000002:head v 28'51, mlcod=27'48)", 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.554773025999999, 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0032001339999999999, 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063161+0000", 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1199999999999999e-07 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063165+0000", 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0210000000000001e-06 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063175+0000", 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9059999999999997e-06 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063182+0000", 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7769999999999997e-06 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063223+0000", 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0883999999999997e-05 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.063235+0000", 2024-05-08T08:25:33.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2764e-05 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.066316+0000", 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030807690000000001 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.066361+0000", 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4801e-05 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:6777 3.10 3:0ebe7e22:::rbd_object_map.119152d8614e:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.071541+0000", 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.546392425000001, 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0026104549999999998, 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 6777 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071541+0000", 2024-05-08T08:25:33.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071541+0000", 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071543+0000", 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1039999999999999e-06 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071548+0000", 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2109999999999997e-06 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071559+0000", 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.168e-05 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071566+0000", 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7530000000000004e-06 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071611+0000", 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4520000000000001e-05 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071683+0000", 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1950000000000001e-05 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.071905+0000", 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0002218 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.072542+0000", 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063779799999999997 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.074072+0000", 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001529519 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.074106+0000", 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4461000000000001e-05 2024-05-08T08:25:33.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.074152+0000", 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5658999999999998e-05 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7101 3.a 3:570608b4:::rbd_data.119120fdbc10.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1310720~19922944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.402864911999998, 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053734150000000003, 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.677 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7101 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215071+0000", 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4210000000000002e-06 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217644+0000", 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025721239999999999 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217664+0000", 2024-05-08T08:25:33.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0492999999999998e-05 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217681+0000", 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6985000000000001e-05 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217759+0000", 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7749000000000004e-05 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217900+0000", 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141361 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.218122+0000", 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022196499999999999 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220140+0000", 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020175129999999999 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220373+0000", 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023305999999999999 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220410+0000", 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7146e-05 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220442+0000", 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2598000000000002e-05 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14271 3.a 3:56b37913:::rbd_data.119174045df.000000000000002e:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:24.891244+0000", 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.7266902099999992, 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040489970000000004, 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.682 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14271 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891244+0000", 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891244+0000", 2024-05-08T08:25:33.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891245+0000", 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4649999999999999e-06 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891268+0000", 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2886000000000001e-05 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891283+0000", 2024-05-08T08:25:33.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4848000000000001e-05 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891307+0000", 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4031000000000001e-05 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891356+0000", 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8696000000000003e-05 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.891452+0000", 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6034999999999998e-05 2024-05-08T08:25:33.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.892116+0000", 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00066427399999999996 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.894982+0000", 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028657869999999998 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.895233+0000", 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025091499999999999 2024-05-08T08:25:33.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.895264+0000", 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1282999999999998e-05 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.895293+0000", 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8776999999999999e-05 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.687 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21104 3.a 3:522bcd46:::rbd_data.1191948f0501.000000000000002f:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.476053+0000", 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.1418804319999998, 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030756070000000002, 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21104 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476053+0000", 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476053+0000", 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476054+0000", 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.82e-07 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476057+0000", 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7480000000000002e-06 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476067+0000", 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0679000000000001e-05 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476073+0000", 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478957+0000", 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028839999999999998 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.479043+0000", 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5500000000000005e-05 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.479129+0000", 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6300999999999999e-05 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21166 3.10 3:0c68b0bf:::rbd_data.1191948f0501.0000000000000069:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:33.691 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.476221+0000", 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.1417126949999998, 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024855939999999998, 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21166 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476221+0000", 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476221+0000", 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476222+0000", 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476224+0000", 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5600000000000001e-06 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476231+0000", 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6560000000000003e-06 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476236+0000", 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1420000000000001e-06 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478567+0000", 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023310869999999999 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478643+0000", 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5512000000000005e-05 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478707+0000", 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4185999999999998e-05 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21167 3.10 3:0c68b0bf:::rbd_data.1191948f0501.0000000000000069:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.1416889770000003, 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0026289339999999999, 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21167 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476245+0000", 2024-05-08T08:25:33.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476250+0000", 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1980000000000003e-06 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476258+0000", 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1620000000000008e-06 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.476263+0000", 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4759999999999997e-06 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478732+0000", 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024690300000000001 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478799+0000", 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6427000000000006e-05 2024-05-08T08:25:33.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.478874+0000", 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5189000000000001e-05 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:22230 3.a 3:545786ae:::rbd_id.image_client.0-clone13:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.082473+0000", 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.5354608490000001, 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002459078, 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.700 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 22230 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082473+0000", 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082473+0000", 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082474+0000", 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.407e-06 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082481+0000", 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4069999999999998e-06 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082493+0000", 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2598e-05 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082504+0000", 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0659000000000001e-05 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082565+0000", 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1308000000000003e-05 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.082573+0000", 2024-05-08T08:25:33.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5769999999999996e-06 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.083879+0000", 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013064109999999999 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.083955+0000", 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5315999999999995e-05 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084059+0000", 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010422 2024-05-08T08:25:33.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084714+0000", 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00065502900000000001 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084878+0000", 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000163695 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084908+0000", 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0361e-05 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.084932+0000", 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4090000000000001e-05 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23737 3.10 3:08319af0:::rbd_data.1191ea0f1b1f.0000000000000009:head [set-alloc-hint object_size 16777216 write_size 16777216,write 8015293~35422] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.115824+0000", 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 2.5021095880000002, 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0026206480000000002, 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:33.706 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23737 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115824+0000", 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115824+0000", 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115826+0000", 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6109999999999999e-06 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115861+0000", 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5024000000000002e-05 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115876+0000", 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4935e-05 2024-05-08T08:25:33.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115885+0000", 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9230000000000008e-06 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115939+0000", 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3755999999999998e-05 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.115943+0000", 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0629999999999999e-06 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.117195+0000", 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00125253 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.117241+0000", 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5358999999999997e-05 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.117315+0000", 2024-05-08T08:25:33.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4561000000000007e-05 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.118080+0000", 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00076539499999999999 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.118393+0000", 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00031272400000000001 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.118423+0000", 2024-05-08T08:25:33.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0178000000000001e-05 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.118445+0000", 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1588999999999999e-05 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:33.712 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:33.713 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:33.714 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:33.720+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:33.814 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:33.821+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:33.861 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:33.861 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:33.861 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:33.861 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:33.870 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:33.955 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:33.962+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:34.016 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:34.022+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:34.017 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:34.017 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:34.017 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:34.017 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:34.017 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:34.026 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:34.067 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:25:34.116 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:34.123+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:34.135 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone14 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.622715993, 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:25:34.182 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:25:34.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:88 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.467230588, 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084184770000000006, 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 88 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:34.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721094+0000", 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1380000000000001e-06 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721098+0000", 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4120000000000001e-06 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721100+0000", 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6910000000000002e-06 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725525+0000", 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044250560000000001 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725656+0000", 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013079000000000001 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725765+0000", 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000108571 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728094+0000", 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023291229999999998 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729456+0000", 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013616780000000001 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729487+0000", 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0956e-05 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729510+0000", 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3711999999999999e-05 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:107 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268232064000003, 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083640399999999997, 2024-05-08T08:25:34.193 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 107 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920090+0000", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920091+0000", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920093+0000", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.661e-06 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920098+0000", 2024-05-08T08:25:34.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2560000000000004e-06 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920101+0000", 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.932e-06 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920638+0000", 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00053755999999999997 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920689+0000", 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0627999999999997e-05 2024-05-08T08:25:34.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920762+0000", 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2768999999999994e-05 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925616+0000", 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048541820000000003 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928394+0000", 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027782010000000001 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928431+0000", 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6915000000000001e-05 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928454+0000", 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3547999999999999e-05 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.198 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268216293999998, 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084564540000000004, 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 108 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920108+0000", 2024-05-08T08:25:34.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9740000000000001e-06 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920112+0000", 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920114+0000", 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6630000000000002e-06 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920996+0000", 2024-05-08T08:25:34.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00088141399999999996 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921044+0000", 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8078000000000003e-05 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921120+0000", 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6469000000000003e-05 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925621+0000", 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045009630000000002 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928490+0000", 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028691929999999999 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928539+0000", 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8857000000000003e-05 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928563+0000", 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3302000000000002e-05 2024-05-08T08:25:34.203 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268204306999998, 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085274359999999993, 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.204 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 109 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1e-07 2024-05-08T08:25:34.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920144+0000", 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.586e-05 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920148+0000", 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1790000000000001e-06 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920151+0000", 2024-05-08T08:25:34.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0359999999999998e-06 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921404+0000", 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00125264 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921448+0000", 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4227000000000002e-05 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921520+0000", 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1532999999999994e-05 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925631+0000", 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041116260000000002 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928593+0000", 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002961881 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928621+0000", 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7259000000000001e-05 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928646+0000", 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4984999999999999e-05 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268163381999997, 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085697380000000004, 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920162+0000", 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920165+0000", 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4640000000000002e-06 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920168+0000", 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6249999999999999e-06 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921849+0000", 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016816789999999999 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921895+0000", 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5198000000000003e-05 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921968+0000", 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3178999999999998e-05 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925638+0000", 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036703859999999999 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928677+0000", 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003038303 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928708+0000", 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1186000000000003e-05 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928729+0000", 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1035000000000001e-05 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268150568000003, 2024-05-08T08:25:34.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086906299999999995, 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 111 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.03e-07 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920174+0000", 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9230000000000001e-06 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920177+0000", 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0539999999999999e-06 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920180+0000", 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8480000000000001e-06 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922200+0000", 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020199060000000001 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922246+0000", 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6319999999999997e-05 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0517000000000003e-05 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925642+0000", 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033256649999999998 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928770+0000", 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031275640000000002 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928829+0000", 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8690000000000002e-05 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928862+0000", 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3939999999999997e-05 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268138721, 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087670669999999999, 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 112 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920214+0000", 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9626999999999999e-05 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920218+0000", 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1320000000000004e-06 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920225+0000", 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5859999999999999e-06 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922574+0000", 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002349009 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922651+0000", 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7014999999999994e-05 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922793+0000", 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141839 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928736+0000", 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005942363 2024-05-08T08:25:34.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928901+0000", 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00016587700000000001 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928930+0000", 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8487e-05 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928951+0000", 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0908e-05 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.224 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268088249999998, 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087948200000000001, 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 113 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.79e-07 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920237+0000", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.368e-06 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920241+0000", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920244+0000", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7549999999999999e-06 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923291+0000", 2024-05-08T08:25:34.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030469809999999998 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923338+0000", 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6501000000000001e-05 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923409+0000", 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1198000000000005e-05 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928739+0000", 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053299819999999996 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928985+0000", 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00024582599999999999 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929010+0000", 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5145999999999999e-05 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929029+0000", 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9184000000000001e-05 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.229 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268073553999997, 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088585829999999997, 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 114 2024-05-08T08:25:34.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:34.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.11e-07 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920251+0000", 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0310000000000001e-06 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920254+0000", 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1549999999999999e-06 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920257+0000", 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7360000000000001e-06 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923648+0000", 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033905189999999998 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923693+0000", 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170999999999998e-05 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923764+0000", 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0875000000000002e-05 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928874+0000", 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051101599999999999 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929058+0000", 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018473 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929087+0000", 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8495000000000001e-05 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929107+0000", 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.05e-05 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268061047000003, 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089168839999999999, 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.235 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 115 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920262+0000", 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920301+0000", 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9572e-05 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920305+0000", 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2390000000000004e-06 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920309+0000", 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.14e-06 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924004+0000", 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036953369999999999 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924048+0000", 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4342999999999999e-05 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924120+0000", 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1489999999999995e-05 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928881+0000", 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047611650000000004 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929135+0000", 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025369599999999998 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929159+0000", 2024-05-08T08:25:34.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4479999999999999e-05 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929178+0000", 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9208999999999999e-05 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.268008420999998, 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089378329999999992, 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.240 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 116 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6199999999999999e-07 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920316+0000", 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1260000000000001e-06 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920326+0000", 2024-05-08T08:25:34.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4229999999999994e-06 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920330+0000", 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924512+0000", 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041817440000000003 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924558+0000", 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6285000000000002e-05 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924639+0000", 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0653000000000002e-05 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928884+0000", 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042451420000000004 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929204+0000", 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032002899999999999 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929232+0000", 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8048e-05 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929252+0000", 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9581000000000001e-05 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.267986123, 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008977779, 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 117 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920337+0000", 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7000000000000001e-07 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920341+0000", 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3039999999999998e-06 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920346+0000", 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920348+0000", 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6809999999999998e-06 2024-05-08T08:25:34.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925057+0000", 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047081240000000002 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925101+0000", 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4838e-05 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925174+0000", 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2619000000000004e-05 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928959+0000", 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037846730000000001 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929272+0000", 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00031341099999999998 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929296+0000", 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4226000000000001e-05 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929314+0000", 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7808999999999998e-05 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 35.571994107000002, 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083370549999999995, 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 139 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616328+0000", 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616329+0000", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0499999999999999e-07 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616332+0000", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8219999999999998e-06 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616336+0000", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1910000000000002e-06 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616339+0000", 2024-05-08T08:25:34.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1650000000000002e-06 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621594+0000", 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052553139999999996 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621649+0000", 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4416000000000002e-05 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621721+0000", 2024-05-08T08:25:34.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1977000000000006e-05 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.623088+0000", 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013677889999999999 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624604+0000", 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0015159279999999999 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624643+0000", 2024-05-08T08:25:34.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8504999999999998e-05 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624665+0000", 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2543000000000001e-05 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 35.571976472999999, 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083691970000000001, 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.256 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 140 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:34.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616346+0000", 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.41e-07 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616374+0000", 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8e-05 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616379+0000", 2024-05-08T08:25:34.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3579999999999996e-06 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616381+0000", 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8389999999999998e-06 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.621985+0000", 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056040489999999998 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622031+0000", 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5683000000000003e-05 2024-05-08T08:25:34.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622098+0000", 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7063000000000005e-05 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.623381+0000", 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001283228 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624673+0000", 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001291295 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624697+0000", 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4122999999999999e-05 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624715+0000", 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8318e-05 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:34.261 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 35.571934364000001, 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098144059999999995, 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 141 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3299999999999998e-07 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616391+0000", 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4360000000000001e-06 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616395+0000", 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.822e-06 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616397+0000", 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.729e-06 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622404+0000", 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0060065120000000003 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622448+0000", 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3707e-05 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622520+0000", 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624923+0000", 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024033819999999999 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626156+0000", 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001232216 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626183+0000", 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7347000000000001e-05 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626202+0000", 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9564000000000001e-05 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:6914 3.13 3:cdf44573:::rbd_data.119120fdbc10.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13546835~4324] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.032440338000001, 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0094012790000000002, 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 6914 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155884+0000", 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7060000000000001e-06 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159051+0000", 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031671109999999998 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159073+0000", 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1667000000000001e-05 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159083+0000", 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0881e-05 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159151+0000", 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7706000000000005e-05 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159252+0000", 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000101098 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159508+0000", 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025555700000000001 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.163927+0000", 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044189390000000002 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165209+0000", 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012823610000000001 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165248+0000", 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9184999999999998e-05 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165283+0000", 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5067999999999997e-05 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14412 3.d 3:b3426b33:::rbd_data.119174045df.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.1641053320000001, 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024946593, 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.272 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14412 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024218+0000", 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024221+0000", 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5489999999999998e-06 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024231+0000", 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.611e-06 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024236+0000", 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1950000000000002e-06 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024278+0000", 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2191000000000003e-05 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024328+0000", 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0008e-05 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.028903+0000", 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045743340000000002 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049023+0000", 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020120815 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049106+0000", 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2900999999999998e-05 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049140+0000", 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4029000000000002e-05 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049164+0000", 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3280000000000001e-05 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23281 3.15 3:aef679f3:::rbd_data.1191ea0f1b1f.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.7118722339999999, 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015244407, 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23281 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476452+0000", 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.562e-06 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483043+0000", 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065909009999999997 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483062+0000", 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9538000000000002e-05 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483074+0000", 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1915e-05 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483133+0000", 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9339000000000002e-05 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483271+0000", 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013750500000000001 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483684+0000", 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0004126 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.486766+0000", 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030822380000000002 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491572+0000", 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048065629999999998 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491643+0000", 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0518000000000005e-05 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491695+0000", 2024-05-08T08:25:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1727999999999997e-05 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23287 3.13 3:ced44c57:::rbd_data.1191ea0f1b1f.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.6752632670000001, 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020994202999999999, 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23287 2024-05-08T08:25:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513061+0000", 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5889999999999999e-06 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524070+0000", 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011009694 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524089+0000", 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8819000000000001e-05 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524099+0000", 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9580000000000002e-06 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524157+0000", 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8106000000000002e-05 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524271+0000", 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000113231 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524698+0000", 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00042707900000000002 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533002+0000", 2024-05-08T08:25:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083041999999999994 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533940+0000", 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00093861900000000002 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534001+0000", 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0492999999999998e-05 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534053+0000", 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2414999999999999e-05 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:34.288 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:34.289 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:34.295+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:34.289 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:34.292 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-05-08T08:25:34.390 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:34.397+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:34.408 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:34.408 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:34.408 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:34.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:34.433 INFO:teuthology.orchestra.run.smithi031.stdout:2654 trunc from 0xd704270 to 0x1790a27 2024-05-08T08:25:34.461 INFO:teuthology.orchestra.run.smithi031.stdout:2655 write 0x5a026e7 thru 0x5a0e8ea (0xc204 bytes) 2024-05-08T08:25:34.478 INFO:teuthology.orchestra.run.smithi031.stdout:2656 writesame 0xcb960b6 thru 0xcb98b73 data_size 0x1(0x2abe bytes) 2024-05-08T08:25:34.478 INFO:teuthology.orchestra.run.smithi031.stdout:2659 read 0xf152e thru 0xf3b14 (0x25e7 bytes) 2024-05-08T08:25:34.478 INFO:teuthology.orchestra.run.smithi031.stdout:2660 write 0xa628554 thru 0xa636565 (0xe012 bytes) 2024-05-08T08:25:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.482 INFO:teuthology.orchestra.run.smithi031.stdout:2666 write 0x565d594 thru 0x566a074 (0xcae1 bytes) 2024-05-08T08:25:34.487 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.487 INFO:teuthology.orchestra.run.smithi031.stdout:2669 read 0x8f0b1db thru 0x8f0b799 (0x5bf bytes) 2024-05-08T08:25:34.487 INFO:teuthology.orchestra.run.smithi031.stdout:2670 writesame 0x9d180a9 thru 0x9d230e1 data_size 0x1(0xb039 bytes) 2024-05-08T08:25:34.491 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:34.497+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:34.507 INFO:teuthology.orchestra.run.smithi031.stdout:2673 read 0xbad9ae8 thru 0xbae66a0 (0xcbb9 bytes) 2024-05-08T08:25:34.507 INFO:teuthology.orchestra.run.smithi031.stdout:2674 read 0xb2e8588 thru 0xb2eb9e2 (0x345b bytes) 2024-05-08T08:25:34.507 INFO:teuthology.orchestra.run.smithi031.stdout:2675 writesame 0x1a383cd thru 0x1a43050 data_size 0x4(0xac84 bytes) 2024-05-08T08:25:34.515 INFO:teuthology.orchestra.run.smithi031.stdout:2676 punch from 0x88d2815 to 0x88df746, (0xcf31 bytes) 2024-05-08T08:25:34.515 INFO:teuthology.orchestra.run.smithi031.stdout:2677 write 0x6123923 thru 0x6133067 (0xf745 bytes) 2024-05-08T08:25:34.519 INFO:teuthology.orchestra.run.smithi031.stdout:2678 writesame 0x3db0e24 thru 0x3dbf3d6 data_size 0x11(0xe5b3 bytes) 2024-05-08T08:25:34.534 INFO:teuthology.orchestra.run.smithi031.stdout:2679 trunc from 0xcb98b74 to 0x2b8aa96 2024-05-08T08:25:34.549 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:34.549 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:34.550 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:34.550 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:34.563 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:34.566 INFO:teuthology.orchestra.run.smithi031.stdout:2680 write 0xbca4652 thru 0xbca6121 (0x1ad0 bytes) 2024-05-08T08:25:34.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.568 INFO:teuthology.orchestra.run.smithi031.stdout:2682 read 0x289b6fe thru 0x28aa641 (0xef44 bytes) 2024-05-08T08:25:34.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.575 INFO:teuthology.orchestra.run.smithi031.stdout:2685 trunc from 0xbca6122 to 0x5c28ae7 2024-05-08T08:25:34.588 INFO:teuthology.orchestra.run.smithi031.stdout:2686 read 0x3561b6f thru 0x3562773 (0xc05 bytes) 2024-05-08T08:25:34.588 INFO:teuthology.orchestra.run.smithi031.stdout:2687 write 0x13fd7d3 thru 0x13fe8ad (0x10db bytes) 2024-05-08T08:25:34.592 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:34.598+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:34.592 INFO:teuthology.orchestra.run.smithi031.stdout:2689 write 0xcef95c thru 0xcfe801 (0xeea6 bytes) 2024-05-08T08:25:34.598 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.605 INFO:teuthology.orchestra.run.smithi031.stdout:2691 writesame 0x6e9fe6c thru 0x6ea099c data_size 0x5(0xb31 bytes) 2024-05-08T08:25:34.605 INFO:teuthology.orchestra.run.smithi031.stdout:2693 read 0x2e79ba5 thru 0x2e83ee2 (0xa33e bytes) 2024-05-08T08:25:34.605 INFO:teuthology.orchestra.run.smithi031.stdout:2694 read 0x1403f thru 0x15617 (0x15d9 bytes) 2024-05-08T08:25:34.618 INFO:teuthology.orchestra.run.smithi031.stdout:2695 write 0xb698b9d thru 0xb6999e9 (0xe4d bytes) 2024-05-08T08:25:34.621 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.621 INFO:teuthology.orchestra.run.smithi031.stdout:2698 trunc from 0xb6999ea to 0xba01c5a 2024-05-08T08:25:34.626 INFO:teuthology.orchestra.run.smithi031.stdout:2700 write 0x76e6e3b thru 0x76e8fd5 (0x219b bytes) 2024-05-08T08:25:34.629 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.629 INFO:teuthology.orchestra.run.smithi031.stdout:2704 write 0xb658b6f thru 0xb65bd90 (0x3222 bytes) 2024-05-08T08:25:34.644 INFO:teuthology.orchestra.run.smithi031.stdout:2707 trunc from 0xba01c5a to 0x1387c03 2024-05-08T08:25:34.677 INFO:teuthology.orchestra.run.smithi031.stdout:2708 trunc from 0x1387c03 to 0xbf1e3ef 2024-05-08T08:25:34.682 INFO:teuthology.orchestra.run.smithi031.stdout:2709 write 0x6f807db thru 0x6f875fd (0x6e23 bytes) 2024-05-08T08:25:34.685 INFO:teuthology.orchestra.run.smithi031.stdout:2710 writesame 0x228c840 thru 0x228fbcb data_size 0x2(0x338c bytes) 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6562 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.763963377, 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024048491000000002, 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6562 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931270+0000", 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.19e-06 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931276+0000", 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4790000000000004e-06 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931286+0000", 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.044e-05 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931292+0000", 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931346+0000", 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4274999999999998e-05 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931398+0000", 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1999999999999997e-05 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.935504+0000", 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041055170000000004 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955221+0000", 2024-05-08T08:25:34.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019717443000000001 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955234+0000", 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2191999999999999e-05 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955271+0000", 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7008000000000002e-05 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955317+0000", 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6796999999999999e-05 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7004 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7077888~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.507013528999998, 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016625803000000001, 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7004 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188222+0000", 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7219999999999999e-06 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188229+0000", 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6359999999999997e-06 2024-05-08T08:25:34.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188234+0000", 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.502e-06 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203959+0000", 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015725330999999999 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204038+0000", 2024-05-08T08:25:34.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8379000000000003e-05 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204845+0000", 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080675500000000004 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7020 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.50692433, 2024-05-08T08:25:34.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017204636999999998, 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7020 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188311+0000", 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188317+0000", 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7959999999999998e-06 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188322+0000", 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204639+0000", 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016316714 2024-05-08T08:25:34.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204730+0000", 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0771000000000002e-05 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205513+0000", 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078277700000000002 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7021 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3145728~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506903352999998, 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017850441000000002, 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7021 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6199999999999999e-07 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188331+0000", 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1069999999999999e-06 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188338+0000", 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188343+0000", 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-05-08T08:25:34.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205565+0000", 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017221841000000002 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205665+0000", 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9530000000000004e-05 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206179+0000", 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051477999999999995 2024-05-08T08:25:34.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7022 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 5767168~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506882232999999, 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018456931999999999, 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7022 2024-05-08T08:25:34.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:34.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188352+0000", 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188358+0000", 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188363+0000", 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206229+0000", 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017865941 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206327+0000", 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7888999999999997e-05 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206807+0000", 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048019000000000002 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7038 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506776329000001, 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019193815999999999, 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7038 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:34.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188457+0000", 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188461+0000", 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188469+0000", 2024-05-08T08:25:34.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188477+0000", 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3559999999999993e-06 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206829+0000", 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018352456 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206928+0000", 2024-05-08T08:25:34.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8981999999999995e-05 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207650+0000", 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072148600000000002 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7039 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506743587999999, 2024-05-08T08:25:34.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491153000000001, 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7039 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188492+0000", 2024-05-08T08:25:34.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188499+0000", 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7469999999999999e-06 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188505+0000", 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207495+0000", 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990683000000001 2024-05-08T08:25:34.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207577+0000", 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.174e-05 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207980+0000", 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040296899999999998 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7040 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506720299000001, 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020064733000000001, 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7040 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188516+0000", 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999999e-06 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188522+0000", 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188527+0000", 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208019+0000", 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491871000000001 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208118+0000", 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8855000000000001e-05 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208577+0000", 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000458784 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7056 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506639476, 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020583127999999999, 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-05-08T08:25:34.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:34.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188595+0000", 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0679999999999999e-06 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188603+0000", 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999997e-06 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188609+0000", 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-05-08T08:25:34.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208623+0000", 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0200141 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208713+0000", 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9806e-05 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209176+0000", 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046324300000000001 2024-05-08T08:25:34.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7057 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4718592~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506586359, 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021149303000000001, 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188647+0000", 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188650+0000", 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7100000000000001e-06 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188657+0000", 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188665+0000", 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4410000000000004e-06 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209211+0000", 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020546426999999999 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209299+0000", 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8139000000000004e-05 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209795+0000", 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049584800000000001 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7058 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7340032~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506556046, 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021706842, 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7058 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:34.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188677+0000", 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188681+0000", 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188688+0000", 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.494e-06 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188693+0000", 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209838+0000", 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021144677000000001 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209901+0000", 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3322000000000001e-05 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210383+0000", 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048203800000000001 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7073 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2359296~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:34.737 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506460536999999, 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022130316000000001, 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188775+0000", 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188782+0000", 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8630000000000008e-06 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188787+0000", 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210421+0000", 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021633454999999999 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210481+0000", 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9911000000000003e-05 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210902+0000", 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042136000000000002 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7074 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4980736~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506438241000001, 2024-05-08T08:25:34.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022761245999999999, 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188795+0000", 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188797+0000", 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-05-08T08:25:34.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188803+0000", 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188807+0000", 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210951+0000", 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022143843 2024-05-08T08:25:34.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211043+0000", 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2044000000000002e-05 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211555+0000", 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051209900000000002 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7075 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7602176~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.506418314000001, 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025123269, 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7075 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188817+0000", 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188823+0000", 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188828+0000", 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-05-08T08:25:34.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211585+0000", 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022757553999999999 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211640+0000", 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000002e-05 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213937+0000", 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002297182 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7090 3.12 3:4c3c5acc:::rbd_data.119120fdbc10.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1572864~19660800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.496944190000001, 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017096892999999998, 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7090 2024-05-08T08:25:34.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198290+0000", 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.666e-06 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198297+0000", 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198308+0000", 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0641e-05 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198314+0000", 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7850000000000003e-06 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213971+0000", 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015656415 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214141+0000", 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016982999999999999 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214349+0000", 2024-05-08T08:25:34.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020874199999999999 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215032+0000", 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068226599999999995 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215223+0000", 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191263 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215286+0000", 2024-05-08T08:25:34.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000004e-05 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215385+0000", 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9161999999999997e-05 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10928 3.6 3:66234790:::rbd_data.11917ede465c.0000000000000000:9 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.556234413, 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016562112, 2024-05-08T08:25:34.755 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10928 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139012+0000", 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3908e-05 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139020+0000", 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.96e-06 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139026+0000", 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5219999999999998e-06 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155334+0000", 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016308699999999999 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155417+0000", 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2229000000000002e-05 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155560+0000", 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014340300000000001 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10943 3.6 3:62dc9057:::rbd_data.119177d1b31a.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.554472427, 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018993326000000001, 2024-05-08T08:25:34.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10943 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140761+0000", 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999999e-07 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140765+0000", 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140775+0000", 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9690000000000006e-06 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140782+0000", 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156122+0000", 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015340006999999999 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156209+0000", 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7249000000000004e-05 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159642+0000", 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034333430000000002 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159700+0000", 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8078999999999997e-05 2024-05-08T08:25:34.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159753+0000", 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3019000000000002e-05 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:13015 3.1 3:828b8991:::rbd_data.1191c87c6f01.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 20971520~6291456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.525182+0000", 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.170050345, 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016529993999999999, 2024-05-08T08:25:34.764 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13015 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.525182+0000", 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.525182+0000", 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.525184+0000", 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527730+0000", 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0025466989999999999 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527754+0000", 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3634000000000001e-05 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527766+0000", 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2228999999999999e-05 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.527848+0000", 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1841000000000005e-05 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.528021+0000", 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017315700000000001 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.528461+0000", 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00043973200000000002 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535762+0000", 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0073014610000000004 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.541578+0000", 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.005815705 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.541655+0000", 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7076000000000005e-05 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.541712+0000", 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6826000000000001e-05 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:13022 3.17 3:ee16c3de:::rbd_data.1191c87c6f01.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 20828828~4380244] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 11.16129797, 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020689712999999998, 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13022 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533938+0000", 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9879999999999998e-06 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534933+0000", 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099514400000000011 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534955+0000", 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1225000000000001e-05 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534965+0000", 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597999999999999e-05 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535030+0000", 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5068999999999997e-05 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535157+0000", 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126501 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535401+0000", 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024435700000000001 2024-05-08T08:25:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.540584+0000", 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051824109999999996 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554524+0000", 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013939890999999999 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554584+0000", 2024-05-08T08:25:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554624+0000", 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0028999999999999e-05 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:23289 3.3 e41/18 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head v 41'115, mlcod=39'113)", 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1715431909999996, 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024290019999999999, 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523690+0000", 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2900000000000002e-07 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545794+0000", 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022104220000000001 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545810+0000", 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5420000000000001e-05 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545818+0000", 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0979999999999994e-06 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545894+0000", 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6579e-05 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545914+0000", 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9372e-05 2024-05-08T08:25:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547942+0000", 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020284220000000002 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547979+0000", 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6980000000000002e-05 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.779 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:34.780 INFO:teuthology.orchestra.run.smithi031.stdout:2711 write 0x766c00a thru 0x766f9f0 (0x39e7 bytes) 2024-05-08T08:25:34.780 INFO:teuthology.orchestra.run.smithi031.stdout:2712 writesame 0xbb3f000 thru 0xbb4dca1 data_size 0x7(0xeca2 bytes) 2024-05-08T08:25:34.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:34.782 INFO:teuthology.orchestra.run.smithi031.stdout:2713 read 0x8221140 thru 0x82283df (0x72a0 bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2714 trunc from 0xbf1e3ef to 0x66cd68d 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2716 write 0xa19a7fd thru 0xa1a9632 (0xee36 bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2717 writesame 0x958101b thru 0x95838a9 data_size 0x1(0x288f bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2719 read 0x1781a61 thru 0x178c01a (0xa5ba bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2721 write 0x77c80a6 thru 0x77ce164 (0x60bf bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2722 read 0x9bd90b4 thru 0x9bdd77d (0x46ca bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2724 punch from 0x1e630f6 to 0x1e6548f, (0x2399 bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2725 punch from 0x458113f to 0x458a657, (0x9518 bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2726 write 0xea8fcbd thru 0xea9626c (0x65b0 bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2727 read 0x379efc5 thru 0x37a435d (0x5399 bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2729 writesame 0xc2ea93 thru 0xc3c59e data_size 0x6(0xdb0c bytes) 2024-05-08T08:25:34.783 INFO:teuthology.orchestra.run.smithi031.stdout:2731 trunc from 0xea9626d to 0x546b5e6 2024-05-08T08:25:34.784 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:34.787+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:34.784 INFO:teuthology.orchestra.run.smithi031.stdout:2732 trunc from 0x546b5e6 to 0x8bc9ee 2024-05-08T08:25:34.807 INFO:teuthology.orchestra.run.smithi031.stdout:2733 punch from 0x7f7bc to 0x84472, (0x4cb6 bytes) 2024-05-08T08:25:34.828 INFO:teuthology.orchestra.run.smithi031.stdout:2734 write 0xbf06697 thru 0xbf0cefb (0x6865 bytes) 2024-05-08T08:25:34.830 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.830 INFO:teuthology.orchestra.run.smithi031.stdout:2738 punch from 0x9a202d4 to 0x9a27e34, (0x7b60 bytes) 2024-05-08T08:25:34.831 INFO:teuthology.orchestra.run.smithi031.stdout:2739 write 0x6015b57 thru 0x6024515 (0xe9bf bytes) 2024-05-08T08:25:34.838 INFO:teuthology.orchestra.run.smithi031.stdout:2740 read 0x6401548 thru 0x64104df (0xef98 bytes) 2024-05-08T08:25:34.838 INFO:teuthology.orchestra.run.smithi031.stdout:2741 read 0x8e2d620 thru 0x8e32852 (0x5233 bytes) 2024-05-08T08:25:34.838 INFO:teuthology.orchestra.run.smithi031.stdout:2742 write 0x549f58b thru 0x54a9be7 (0xa65d bytes) 2024-05-08T08:25:34.843 INFO:teuthology.orchestra.run.smithi031.stdout:2743 read 0x853dceb thru 0x853e07d (0x393 bytes) 2024-05-08T08:25:34.844 INFO:teuthology.orchestra.run.smithi031.stdout:2744 read 0x8218b67 thru 0x821c0c8 (0x3562 bytes) 2024-05-08T08:25:34.844 INFO:teuthology.orchestra.run.smithi031.stdout:2745 write 0x92a660a thru 0x92aa0b7 (0x3aae bytes) 2024-05-08T08:25:34.846 INFO:teuthology.orchestra.run.smithi031.stdout:2746 writesame 0x2c812a8 thru 0x2c89c8a data_size 0x1(0x89e3 bytes) 2024-05-08T08:25:34.864 INFO:teuthology.orchestra.run.smithi031.stdout:2747 write 0xb457b58 thru 0xb45db2c (0x5fd5 bytes) 2024-05-08T08:25:34.870 INFO:teuthology.orchestra.run.smithi031.stdout:2748 trunc from 0xbf0cefc to 0xded7ce0 2024-05-08T08:25:34.875 INFO:teuthology.orchestra.run.smithi031.stdout:2749 read 0xb1ae123 thru 0xb1ae627 (0x505 bytes) 2024-05-08T08:25:34.875 INFO:teuthology.orchestra.run.smithi031.stdout:2750 writesame 0x7de2df5 thru 0x7df1f00 data_size 0x2(0xf10c bytes) 2024-05-08T08:25:34.881 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:34.888+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:34.891 INFO:teuthology.orchestra.run.smithi031.stdout:2753 write 0x78d3ee0 thru 0x78dfb0c (0xbc2d bytes) 2024-05-08T08:25:34.894 INFO:teuthology.orchestra.run.smithi031.stdout:2754 write 0xdd4fbfa thru 0xdd5ee6f (0xf276 bytes) 2024-05-08T08:25:34.897 INFO:teuthology.orchestra.run.smithi031.stdout:2755 write 0x1acd54b thru 0x1adcfc5 (0xfa7b bytes) 2024-05-08T08:25:34.900 INFO:teuthology.orchestra.run.smithi031.stdout:2756 write 0x182a74e thru 0x182b7b9 (0x106c bytes) 2024-05-08T08:25:34.903 INFO:teuthology.orchestra.run.smithi031.stdout:2758 write 0x2741fcd thru 0x274375e (0x1792 bytes) 2024-05-08T08:25:34.905 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:34.905 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:25356 3.4 3:246d6bfa:::rbd_data.11911e381319.0000000000000053:head [set-alloc-hint object_size 524288 write_size 524288,write 8141~6034 in=6034b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:34.913108+0000", 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00029382799999999999, 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00031381600000000002, 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:34.906 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 25356 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913108+0000", 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913108+0000", 2024-05-08T08:25:34.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913110+0000", 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4670000000000001e-06 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913119+0000", 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3810000000000005e-06 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913132+0000", 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3151e-05 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913139+0000", 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8820000000000003e-06 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913197+0000", 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7500000000000002e-05 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.913290+0000", 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2831999999999995e-05 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:34.911 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:34.911 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-05-08T08:25:34.911 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:34.911 INFO:teuthology.orchestra.run.smithi031.stdout:2760 punch from 0xd8e7370 to 0xd8ea585, (0x3215 bytes) 2024-05-08T08:25:34.911 INFO:teuthology.orchestra.run.smithi031.stdout:2761 read 0x79e9716 thru 0x79f2d77 (0x9662 bytes) 2024-05-08T08:25:34.912 INFO:teuthology.orchestra.run.smithi031.stdout:2763 trunc from 0xded7ce0 to 0x983493b 2024-05-08T08:25:34.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:34.921 INFO:teuthology.orchestra.run.smithi031.stdout:2764 write 0x3d3fcd7 thru 0x3d4ce7e (0xd1a8 bytes) 2024-05-08T08:25:34.926 INFO:teuthology.orchestra.run.smithi031.stdout:2765 write 0x5634992 thru 0x5643571 (0xebe0 bytes) 2024-05-08T08:25:34.931 INFO:teuthology.orchestra.run.smithi031.stdout:2766 read 0x3c4dc5a thru 0x3c587b8 (0xab5f bytes) 2024-05-08T08:25:34.931 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.932 INFO:teuthology.orchestra.run.smithi031.stdout:2768 read 0x38716d3 thru 0x3879c28 (0x8556 bytes) 2024-05-08T08:25:34.932 INFO:teuthology.orchestra.run.smithi031.stdout:2769 trunc from 0x983493b to 0xa39bae2 2024-05-08T08:25:34.938 INFO:teuthology.orchestra.run.smithi031.stdout:2770 read 0x96fa35b thru 0x97083e2 (0xe088 bytes) 2024-05-08T08:25:34.938 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.938 INFO:teuthology.orchestra.run.smithi031.stdout:2773 writesame 0x8aacd1c thru 0x8aba6d0 data_size 0x1(0xd9b5 bytes) 2024-05-08T08:25:34.960 INFO:teuthology.orchestra.run.smithi031.stdout:2774 read 0x57fa6d0 thru 0x57fe787 (0x40b8 bytes) 2024-05-08T08:25:34.960 INFO:teuthology.orchestra.run.smithi031.stdout:2775 read 0x91c1506 thru 0x91c61e5 (0x4ce0 bytes) 2024-05-08T08:25:34.961 INFO:teuthology.orchestra.run.smithi031.stdout:2776 punch from 0x65091d to 0x65f210, (0xe8f3 bytes) 2024-05-08T08:25:34.961 INFO:teuthology.orchestra.run.smithi031.stdout:2777 writesame 0x4dde2ba thru 0x4de5c8f data_size 0x5(0x79d6 bytes) 2024-05-08T08:25:34.967 INFO:teuthology.orchestra.run.smithi031.stdout:2779 trunc from 0xa39bae2 to 0x93beadc 2024-05-08T08:25:34.982 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:34.990+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:34.983 INFO:teuthology.orchestra.run.smithi031.stdout:2780 write 0xdd4426c thru 0xdd44646 (0x3db bytes) 2024-05-08T08:25:34.987 INFO:teuthology.orchestra.run.smithi031.stdout:2781 read 0x4ea6f05 thru 0x4ea8564 (0x1660 bytes) 2024-05-08T08:25:34.987 INFO:teuthology.orchestra.run.smithi031.stdout:2782 write 0x2e2e44a thru 0x2e37b2d (0x96e4 bytes) 2024-05-08T08:25:34.990 INFO:teuthology.orchestra.run.smithi031.stdout:2783 read 0x3fa87b1 thru 0x3fa97aa (0xffa bytes) 2024-05-08T08:25:34.990 INFO:teuthology.orchestra.run.smithi031.stdout:2784 read 0xc13b86 thru 0xc1411a (0x595 bytes) 2024-05-08T08:25:34.990 INFO:teuthology.orchestra.run.smithi031.stdout:2785 punch from 0x64c8f31 to 0x64d29ae, (0x9a7d bytes) 2024-05-08T08:25:34.990 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.990 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:34.990 INFO:teuthology.orchestra.run.smithi031.stdout:2788 writesame 0xd7b64d thru 0xd83d4e data_size 0xb(0x8702 bytes) 2024-05-08T08:25:34.994 INFO:teuthology.orchestra.run.smithi031.stdout:2790 punch from 0x5b9a874 to 0x5b9d44c, (0x2bd8 bytes) 2024-05-08T08:25:34.994 INFO:teuthology.orchestra.run.smithi031.stdout:2792 read 0x13b456d thru 0x13b8e68 (0x48fc bytes) 2024-05-08T08:25:34.994 INFO:teuthology.orchestra.run.smithi031.stdout:2794 read 0x58d9a60 thru 0x58e5bd8 (0xc179 bytes) 2024-05-08T08:25:34.994 INFO:teuthology.orchestra.run.smithi031.stdout:2795 write 0x89428db thru 0x894a277 (0x799d bytes) 2024-05-08T08:25:34.997 INFO:teuthology.orchestra.run.smithi031.stdout:2797 read 0xa04b498 thru 0xa051d86 (0x68ef bytes) 2024-05-08T08:25:34.997 INFO:teuthology.orchestra.run.smithi031.stdout:2798 read 0x8eea9d8 thru 0x8ef32b5 (0x88de bytes) 2024-05-08T08:25:34.997 INFO:teuthology.orchestra.run.smithi031.stdout:2799 write 0x57786d5 thru 0x5782930 (0xa25c bytes) 2024-05-08T08:25:35.000 INFO:teuthology.orchestra.run.smithi031.stdout:2800 read 0x2f6958e thru 0x2f74f75 (0xb9e8 bytes) 2024-05-08T08:25:35.001 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:35.001 INFO:teuthology.orchestra.run.smithi031.stdout:2802 write 0x549fc9f thru 0x54aed05 (0xf067 bytes) 2024-05-08T08:25:35.003 INFO:teuthology.orchestra.run.smithi031.stdout:2804 punch from 0x832a122 to 0x8335ade, (0xb9bc bytes) 2024-05-08T08:25:35.003 INFO:teuthology.orchestra.run.smithi031.stdout:2805 write 0x4eaa7ca thru 0x4eb8bc6 (0xe3fd bytes) 2024-05-08T08:25:35.004 INFO:teuthology.orchestra.run.smithi031.stdout:2806 punch from 0xd7da89b to 0xd7e4d96, (0xa4fb bytes) 2024-05-08T08:25:35.014 INFO:teuthology.orchestra.run.smithi031.stdout:2807 trunc from 0xdd44647 to 0x522b4c4 2024-05-08T08:25:35.026 INFO:teuthology.orchestra.run.smithi031.stdout:2808 writesame 0x38e153d thru 0x38ed798 data_size 0xe(0xc25c bytes) 2024-05-08T08:25:35.031 INFO:teuthology.orchestra.run.smithi031.stdout:2810 trunc from 0x522b4c4 to 0x49e3131 2024-05-08T08:25:35.040 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:35.040 INFO:teuthology.orchestra.run.smithi031.stdout:2812 write 0x3b1d289 thru 0x3b2c7c1 (0xf539 bytes) 2024-05-08T08:25:35.043 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:35.043 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:35.043 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:35.049 INFO:teuthology.orchestra.run.smithi031.stdout:2813 writesame 0x4d205fe thru 0x4d2c014 data_size 0x1(0xba17 bytes) 2024-05-08T08:25:35.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:35.060 INFO:teuthology.orchestra.run.smithi031.stdout:2816 writesame 0x8cb0b1c thru 0x8cb3fc3 data_size 0x14(0x34a8 bytes) 2024-05-08T08:25:35.067 INFO:teuthology.orchestra.run.smithi031.stdout:2817 write 0xa599952 thru 0xa5a1724 (0x7dd3 bytes) 2024-05-08T08:25:35.074 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:35.074 INFO:teuthology.orchestra.run.smithi031.stdout:2819 write 0x2d79524 thru 0x2d89227 (0xfd04 bytes) 2024-05-08T08:25:35.083 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:35.091+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:35.084 INFO:teuthology.orchestra.run.smithi031.stdout:2820 trunc from 0xa5a1725 to 0x72ee7ee 2024-05-08T08:25:35.101 INFO:teuthology.orchestra.run.smithi031.stdout:2821 writesame 0x7e3ddc4 thru 0x7e4bf7e data_size 0x1(0xe1bb bytes) 2024-05-08T08:25:35.101 INFO:teuthology.orchestra.run.smithi031.stdout:2822 punch from 0x2d78aa0 to 0x2d7ca27, (0x3f87 bytes) 2024-05-08T08:25:35.113 INFO:teuthology.orchestra.run.smithi031.stdout:2823 write 0xdbd9b84 thru 0xdbe8f35 (0xf3b2 bytes) 2024-05-08T08:25:35.119 INFO:teuthology.orchestra.run.smithi031.stdout:2824 read 0x8877850 thru 0x887d308 (0x5ab9 bytes) 2024-05-08T08:25:35.119 INFO:teuthology.orchestra.run.smithi031.stdout:2825 write 0x4e5928 thru 0x4e5a02 (0xdb bytes) 2024-05-08T08:25:35.125 INFO:teuthology.orchestra.run.smithi031.stdout:2826 writesame 0xa0d50e3 thru 0xa0d9151 data_size 0x1(0x406f bytes) 2024-05-08T08:25:35.134 INFO:teuthology.orchestra.run.smithi031.stdout:2830 write 0x72ec80d thru 0x72fbbc8 (0xf3bc bytes) 2024-05-08T08:25:35.137 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:35.137 INFO:teuthology.orchestra.run.smithi031.stdout:2832 writesame 0xa159a53 thru 0xa15a748 data_size 0x15(0xcf6 bytes) 2024-05-08T08:25:35.140 INFO:teuthology.orchestra.run.smithi031.stdout:2833 clone 16 order 18 su 65536 sc 7 2024-05-08T08:25:35.179 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:35.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2370 3.4 e24/18 3:237be9c1:::rbd_data.119116cc7dd0.00000000000001eb:head v 24'46, mlcod=23'42)", 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.464392007000001, 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029578109999999999, 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721287+0000", 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7199999999999997e-07 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721294+0000", 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9800000000000001e-06 2024-05-08T08:25:35.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721309+0000", 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5116999999999999e-05 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9030000000000004e-06 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721383+0000", 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6111000000000005e-05 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721406+0000", 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2453e-05 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724192+0000", 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027864909999999999 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724244+0000", 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.4601787, 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:25:35.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:25:35.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.190485163000002, 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:25:35.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:25:35.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.161397397, 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:25:35.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.046245701, 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.197 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:35.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:25:35.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9225992789999999, 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9225281320000001, 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:25:35.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:35.212 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9221824019999998, 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:35.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9221411130000003, 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:35.217 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20624 3.4 3:21e3449a:::rbd_data.119174045df.0000000000000045:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:35.221 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9218427939999998, 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029912699999999999, 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0200000000000001e-07 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263841+0000", 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:25:35.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263854+0000", 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2937e-05 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263862+0000", 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6729999999999992e-06 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263949+0000", 2024-05-08T08:25:35.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7406999999999998e-05 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263954+0000", 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266741+0000", 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027867640000000002 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266827+0000", 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5780999999999997e-05 2024-05-08T08:25:35.225 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20845 3.0 3:04dbc92b:::rbd_data.1191948f0501.0000000000000008:e [read 413696~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.472101+0000", 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7135779449999999, 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002772133, 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.226 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20845 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472101+0000", 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472101+0000", 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472101+0000", 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.15e-07 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472103+0000", 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.385e-06 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472110+0000", 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2319999999999998e-06 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472115+0000", 2024-05-08T08:25:35.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.474742+0000", 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0026270740000000001 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.474798+0000", 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6372999999999997e-05 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.474873+0000", 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.229 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20847 3.0 3:04dbc92b:::rbd_data.1191948f0501.0000000000000008:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7135576749999997, 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029128679999999999, 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.230 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20847 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6399999999999998e-07 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472123+0000", 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.841e-06 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472128+0000", 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4280000000000004e-06 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472133+0000", 2024-05-08T08:25:35.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3150000000000002e-06 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.474903+0000", 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027698620000000001 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.474958+0000", 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5067e-05 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.475034+0000", 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5990999999999998e-05 2024-05-08T08:25:35.233 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21116 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 339968~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477570+0000", 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7081083000000001, 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027748040000000001, 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.234 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21116 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477570+0000", 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477570+0000", 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477572+0000", 2024-05-08T08:25:35.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.809e-06 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477579+0000", 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7020000000000004e-06 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477584+0000", 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477588+0000", 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6149999999999999e-06 2024-05-08T08:25:35.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480189+0000", 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002600158 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480243+0000", 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4629999999999997e-05 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480345+0000", 2024-05-08T08:25:35.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010183099999999999 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21117 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 413696~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7080838639999998, 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003161652, 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.238 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21117 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477596+0000", 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8050000000000001e-06 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477599+0000", 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.616e-06 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477604+0000", 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477608+0000", 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:35.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480366+0000", 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027578020000000002 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480420+0000", 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3735e-05 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480756+0000", 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033637199999999998 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.7080610060000003, 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:25:35.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:25:35.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:25:35.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:25:35.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21737 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 376832~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624108+0000", 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5615704570000002, 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028862470000000002, 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.246 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21737 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624108+0000", 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624108+0000", 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624109+0000", 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624111+0000", 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624118+0000", 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624123+0000", 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2900000000000002e-06 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.626801+0000", 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002677184 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.626870+0000", 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9692999999999998e-05 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.626994+0000", 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012398699999999999 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21738 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 450560~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.5615478490000001, 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031591710000000001, 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21738 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624133+0000", 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624140+0000", 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624145+0000", 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627084+0000", 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029395129999999999 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627161+0000", 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7096999999999997e-05 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627290+0000", 2024-05-08T08:25:35.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012866399999999999 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.4158471759999998, 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.254 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:25:35.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:25:35.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6886451400000002, 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:25:35.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:25:35.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.469664775, 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:35.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:35.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:25:35.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.267 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:35.268 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:35.269 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:35.269 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:35.276+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:35.369 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:35.376+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:35.417 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:35.420 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:35.420 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:35.420 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:35.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:35.470 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:35.478+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:35.544 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:35.544 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:35.562 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:35.563 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:35.571 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:35.578+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:35.579 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:35.580 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:35.580 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:35.580 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:35.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:35.598 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:35.598 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:35.615 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:35.615 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:35.633 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:35.633 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:35.651 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:35.652 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:35.655 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:35.655 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:35.671 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:35.679+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:35.707 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:35.707 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:35.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:35.772 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:35.780+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:35.826 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:35.826 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:35.834 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:35.877 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:35.885+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:35.939 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:35.939 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:35.948 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:35.989 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:35.997+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:36.075 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:36.081+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:36.087 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:25:36.096 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:36.096 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:36.106 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:36.142 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone15 from 0xdbe8f36 (overlap 0x8bc9ee) to 0x1bf52f 2024-05-08T08:25:36.190 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:36.197+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:36.233 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-05-08T08:25:36.264 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:36.264 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:36.273 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:36.314 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:36.322+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:36.378 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:36.386+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:36.421 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:36.421 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:36.430 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:36.471 INFO:teuthology.orchestra.run.smithi031.stdout:2835 write 0x7808e9a thru 0x7812dcc (0x9f33 bytes) 2024-05-08T08:25:36.479 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:36.487+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:36.480 INFO:teuthology.orchestra.run.smithi031.stdout:2836 writesame 0x3ba93c8 thru 0x3bad0ec data_size 0x1(0x3d25 bytes) 2024-05-08T08:25:36.487 INFO:teuthology.orchestra.run.smithi031.stdout:2837 writesame 0x5dace97 thru 0x5db9866 data_size 0x8(0xc9d0 bytes) 2024-05-08T08:25:36.492 INFO:teuthology.orchestra.run.smithi031.stdout:2840 read 0x6ad3973 thru 0x6ad6a79 (0x3107 bytes) 2024-05-08T08:25:36.493 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.493 INFO:teuthology.orchestra.run.smithi031.stdout:2842 writesame 0x5777e85 thru 0x57782f6 data_size 0x1(0x472 bytes) 2024-05-08T08:25:36.495 INFO:teuthology.orchestra.run.smithi031.stdout:2843 write 0x16f80a7 thru 0x16f8799 (0x6f3 bytes) 2024-05-08T08:25:36.498 INFO:teuthology.orchestra.run.smithi031.stdout:2844 writesame 0x7e3b227 thru 0x7e3ca84 data_size 0x1(0x185e bytes) 2024-05-08T08:25:36.503 INFO:teuthology.orchestra.run.smithi031.stdout:2846 write 0x2064795 thru 0x20699f6 (0x5262 bytes) 2024-05-08T08:25:36.514 INFO:teuthology.orchestra.run.smithi031.stdout:2847 trunc from 0xdbe8f36 to 0x6bf6c27 2024-05-08T08:25:36.536 INFO:teuthology.orchestra.run.smithi031.stdout:2848 read 0x22933b2 thru 0x229dc74 (0xa8c3 bytes) 2024-05-08T08:25:36.536 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.543 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:36.543 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:36.546 INFO:teuthology.orchestra.run.smithi031.stdout:2850 write 0xaf8c3c5 thru 0xaf8fed4 (0x3b10 bytes) 2024-05-08T08:25:36.550 INFO:teuthology.orchestra.run.smithi031.stdout:2851 writesame 0x89c16df thru 0x89cd85a data_size 0x7a(0xc17c bytes) 2024-05-08T08:25:36.553 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:36.559 INFO:teuthology.orchestra.run.smithi031.stdout:2855 write 0xb60cecf thru 0xb61c406 (0xf538 bytes) 2024-05-08T08:25:36.565 INFO:teuthology.orchestra.run.smithi031.stdout:2856 write 0xa6701fc thru 0xa67bf59 (0xbd5e bytes) 2024-05-08T08:25:36.568 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.575 INFO:teuthology.orchestra.run.smithi031.stdout:2858 trunc from 0xb61c407 to 0x583929c 2024-05-08T08:25:36.596 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:36.604+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:36.603 INFO:teuthology.orchestra.run.smithi031.stdout:2859 write 0x694217b thru 0x6943195 (0x101b bytes) 2024-05-08T08:25:36.605 INFO:teuthology.orchestra.run.smithi031.stdout:2860 write 0x395e84b thru 0x39667c9 (0x7f7f bytes) 2024-05-08T08:25:36.610 INFO:teuthology.orchestra.run.smithi031.stdout:2862 read 0x38d8573 thru 0x38e5a70 (0xd4fe bytes) 2024-05-08T08:25:36.611 INFO:teuthology.orchestra.run.smithi031.stdout:2864 write 0x260dc26 thru 0x26166c1 (0x8a9c bytes) 2024-05-08T08:25:36.615 INFO:teuthology.orchestra.run.smithi031.stdout:2865 punch from 0x633e0a to 0x64021f, (0xc415 bytes) 2024-05-08T08:25:36.615 INFO:teuthology.orchestra.run.smithi031.stdout:2866 punch from 0x39df86a to 0x39e4ef4, (0x568a bytes) 2024-05-08T08:25:36.615 INFO:teuthology.orchestra.run.smithi031.stdout:2867 punch from 0x1e107e0 to 0x1e1bcbf, (0xb4df bytes) 2024-05-08T08:25:36.628 INFO:teuthology.orchestra.run.smithi031.stdout:2869 write 0xce89b5c thru 0xce90ac7 (0x6f6c bytes) 2024-05-08T08:25:36.633 INFO:teuthology.orchestra.run.smithi031.stdout:2870 read 0xad385 thru 0xaee60 (0x1adc bytes) 2024-05-08T08:25:36.633 INFO:teuthology.orchestra.run.smithi031.stdout:2871 punch from 0xb1c482a to 0xb1c85d6, (0x3dac bytes) 2024-05-08T08:25:36.637 INFO:teuthology.orchestra.run.smithi031.stdout:2872 trunc from 0xce90ac8 to 0x9811aa3 2024-05-08T08:25:36.656 INFO:teuthology.orchestra.run.smithi031.stdout:2873 writesame 0xaaeb405 thru 0xaaee327 data_size 0x1(0x2f23 bytes) 2024-05-08T08:25:36.656 INFO:teuthology.orchestra.run.smithi031.stdout:2875 write 0x167f2e3 thru 0x1685906 (0x6624 bytes) 2024-05-08T08:25:36.668 INFO:teuthology.orchestra.run.smithi031.stdout:2877 trunc from 0xaaee328 to 0x29cd043 2024-05-08T08:25:36.673 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:36.673 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:36.681 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:36.688+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:36.684 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:36.692 INFO:teuthology.orchestra.run.smithi031.stdout:2878 trunc from 0x29cd043 to 0x8eba689 2024-05-08T08:25:36.698 INFO:teuthology.orchestra.run.smithi031.stdout:2879 trunc from 0x8eba689 to 0x8e80a46 2024-05-08T08:25:36.709 INFO:teuthology.orchestra.run.smithi031.stdout:2882 trunc from 0x8e80a46 to 0xdb62a5f 2024-05-08T08:25:36.715 INFO:teuthology.orchestra.run.smithi031.stdout:2883 write 0xd319d21 thru 0xd321df7 (0x80d7 bytes) 2024-05-08T08:25:36.721 INFO:teuthology.orchestra.run.smithi031.stdout:2885 trunc from 0xdb62a5f to 0xb1f01f7 2024-05-08T08:25:36.731 INFO:teuthology.orchestra.run.smithi031.stdout:2886 writesame 0x29248d0 thru 0x29253e2 data_size 0x9(0xb13 bytes) 2024-05-08T08:25:36.734 INFO:teuthology.orchestra.run.smithi031.stdout:2888 punch from 0x676a842 to 0x67700ed, (0x58ab bytes) 2024-05-08T08:25:36.738 INFO:teuthology.orchestra.run.smithi031.stdout:2890 trunc from 0xb1f01f7 to 0xeaddd34 2024-05-08T08:25:36.745 INFO:teuthology.orchestra.run.smithi031.stdout:2891 punch from 0x334bdc9 to 0x33513f2, (0x5629 bytes) 2024-05-08T08:25:36.745 INFO:teuthology.orchestra.run.smithi031.stdout:2892 writesame 0x4721ed5 thru 0x472850a data_size 0x62(0x6636 bytes) 2024-05-08T08:25:36.748 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.755 INFO:teuthology.orchestra.run.smithi031.stdout:2895 trunc from 0xeaddd34 to 0x8cc6b00 2024-05-08T08:25:36.761 INFO:teuthology.orchestra.run.smithi031.stdout:2896 write 0x6322228 thru 0x63234e5 (0x12be bytes) 2024-05-08T08:25:36.763 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.763 INFO:teuthology.orchestra.run.smithi031.stdout:2898 punch from 0x11374a9 to 0x1138abe, (0x1615 bytes) 2024-05-08T08:25:36.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.773 INFO:teuthology.orchestra.run.smithi031.stdout:2903 writesame 0xbdb189c thru 0xbdc02ee data_size 0x1(0xea53 bytes) 2024-05-08T08:25:36.773 INFO:teuthology.orchestra.run.smithi031.stdout:2905 write 0xa2c4621 thru 0xa2c7cac (0x368c bytes) 2024-05-08T08:25:36.776 INFO:teuthology.orchestra.run.smithi031.stdout:2906 read 0x970ed53 thru 0x970f4a2 (0x750 bytes) 2024-05-08T08:25:36.776 INFO:teuthology.orchestra.run.smithi031.stdout:2908 read 0x3b9bc04 thru 0x3ba3073 (0x7470 bytes) 2024-05-08T08:25:36.776 INFO:teuthology.orchestra.run.smithi031.stdout:2909 write 0xa569291 thru 0xa56b514 (0x2284 bytes) 2024-05-08T08:25:36.779 INFO:teuthology.orchestra.run.smithi031.stdout:2910 writesame 0x58798b6 thru 0x58895f8 data_size 0x1(0xfd43 bytes) 2024-05-08T08:25:36.782 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:36.789+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:36.796 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.798 INFO:teuthology.orchestra.run.smithi031.stdout:2913 trunc from 0xbdc02ef to 0xd844e6c 2024-05-08T08:25:36.805 INFO:teuthology.orchestra.run.smithi031.stdout:2914 read 0xbe784c6 thru 0xbe80821 (0x835c bytes) 2024-05-08T08:25:36.820 INFO:teuthology.orchestra.run.smithi031.stdout:2915 trunc from 0xd844e6c to 0x4e1ca6 2024-05-08T08:25:36.839 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:36.839 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:36.844 INFO:teuthology.orchestra.run.smithi031.stdout:2916 read 0x1c09aa thru 0x1cec9c (0xe2f3 bytes) 2024-05-08T08:25:36.845 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.849 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:36.860 INFO:teuthology.orchestra.run.smithi031.stdout:2919 write 0x8f0c726 thru 0x8f1a034 (0xd90f bytes) 2024-05-08T08:25:36.866 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.872 INFO:teuthology.orchestra.run.smithi031.stdout:2922 write 0x983d5a4 thru 0x98492a7 (0xbd04 bytes) 2024-05-08T08:25:36.877 INFO:teuthology.orchestra.run.smithi031.stdout:2925 punch from 0x15024a0 to 0x151089f, (0xe3ff bytes) 2024-05-08T08:25:36.879 INFO:teuthology.orchestra.run.smithi031.stdout:2926 trunc from 0x98492a8 to 0x7fedc95 2024-05-08T08:25:36.891 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:36.899+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:36.901 INFO:teuthology.orchestra.run.smithi031.stdout:2927 writesame 0xa3f7e4d thru 0xa4045ce data_size 0x1(0xc782 bytes) 2024-05-08T08:25:36.910 INFO:teuthology.orchestra.run.smithi031.stdout:2929 writesame 0xc6d7ed2 thru 0xc6de5aa data_size 0x1(0x66d9 bytes) 2024-05-08T08:25:36.910 INFO:teuthology.orchestra.run.smithi031.stdout:2930 punch from 0x3e49873 to 0x3e581ca, (0xe957 bytes) 2024-05-08T08:25:36.910 INFO:teuthology.orchestra.run.smithi031.stdout:2931 writesame 0x1dff5b4 thru 0x1e0b057 data_size 0xa(0xbaa4 bytes) 2024-05-08T08:25:36.916 INFO:teuthology.orchestra.run.smithi031.stdout:2932 punch from 0x9581f6d to 0x959187f, (0xf912 bytes) 2024-05-08T08:25:36.925 INFO:teuthology.orchestra.run.smithi031.stdout:2933 write 0xe998a62 thru 0xe99add4 (0x2373 bytes) 2024-05-08T08:25:36.929 INFO:teuthology.orchestra.run.smithi031.stdout:2935 write 0x7ac85c8 thru 0x7ad4735 (0xc16e bytes) 2024-05-08T08:25:36.943 INFO:teuthology.orchestra.run.smithi031.stdout:2936 trunc from 0xe99add5 to 0x7177f4e 2024-05-08T08:25:36.955 INFO:teuthology.orchestra.run.smithi031.stdout:2937 writesame 0x4d61341 thru 0x4d61d4d data_size 0x1(0xa0d bytes) 2024-05-08T08:25:36.966 INFO:teuthology.orchestra.run.smithi031.stdout:2938 write 0x7ad7356 thru 0x7ae5329 (0xdfd4 bytes) 2024-05-08T08:25:36.970 INFO:teuthology.orchestra.run.smithi031.stdout:2940 write 0x77e50e6 thru 0x77e6f5b (0x1e76 bytes) 2024-05-08T08:25:36.973 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.980 INFO:teuthology.orchestra.run.smithi031.stdout:2943 write 0x9146add thru 0x9155834 (0xed58 bytes) 2024-05-08T08:25:36.983 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:36.991+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:36.987 INFO:teuthology.orchestra.run.smithi031.stdout:2945 write 0x1355533 thru 0x13644e8 (0xefb6 bytes) 2024-05-08T08:25:36.994 INFO:teuthology.orchestra.run.smithi031.stdout:2947 read 0x8b62ada thru 0x8b6eb94 (0xc0bb bytes) 2024-05-08T08:25:36.994 INFO:teuthology.orchestra.run.smithi031.stdout:2950 punch from 0x5709f51 to 0x570d158, (0x3207 bytes) 2024-05-08T08:25:36.994 INFO:teuthology.orchestra.run.smithi031.stdout:2951 read 0x4517af7 thru 0x45205f6 (0x8b00 bytes) 2024-05-08T08:25:36.995 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:36.995 INFO:teuthology.orchestra.run.smithi031.stdout:2953 read 0x1e95384 thru 0x1e9ebdd (0x985a bytes) 2024-05-08T08:25:36.998 INFO:teuthology.orchestra.run.smithi031.stdout:2954 trunc from 0x9155835 to 0x5915b0b 2024-05-08T08:25:37.003 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:37.003 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:37.011 INFO:teuthology.orchestra.run.smithi031.stdout:2956 punch from 0x489b06b to 0x48aaa97, (0xfa2c bytes) 2024-05-08T08:25:37.012 INFO:teuthology.orchestra.run.smithi031.stdout:2958 trunc from 0x5915b0b to 0x4473580 2024-05-08T08:25:37.013 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:37.036 INFO:teuthology.orchestra.run.smithi031.stdout:2959 write 0x9b60c38 thru 0x9b65b72 (0x4f3b bytes) 2024-05-08T08:25:37.048 INFO:teuthology.orchestra.run.smithi031.stdout:2960 writesame 0xc140b20 thru 0xc14e23a data_size 0x1(0xd71b bytes) 2024-05-08T08:25:37.048 INFO:teuthology.orchestra.run.smithi031.stdout:2961 write 0x8b0228c thru 0x8b027ce (0x543 bytes) 2024-05-08T08:25:37.050 INFO:teuthology.orchestra.run.smithi031.stdout:2962 read 0x985c50a thru 0x985ce2b (0x922 bytes) 2024-05-08T08:25:37.052 INFO:teuthology.orchestra.run.smithi031.stdout:2963 trunc from 0xc14e23b to 0xad6ebcc 2024-05-08T08:25:37.060 INFO:teuthology.orchestra.run.smithi031.stdout:2965 trunc from 0xad6ebcc to 0x8d2a53d 2024-05-08T08:25:37.070 INFO:teuthology.orchestra.run.smithi031.stdout:2966 punch from 0x6e08d1 to 0x6eaaa2, (0xa1d1 bytes) 2024-05-08T08:25:37.072 INFO:teuthology.orchestra.run.smithi031.stdout:2968 trunc from 0x8d2a53d to 0xadaf6f7 2024-05-08T08:25:37.079 INFO:teuthology.orchestra.run.smithi031.stdout:2969 read 0x690daf0 thru 0x6916a5b (0x8f6c bytes) 2024-05-08T08:25:37.079 INFO:teuthology.orchestra.run.smithi031.stdout:2971 read 0x908575 thru 0x914ab6 (0xc542 bytes) 2024-05-08T08:25:37.079 INFO:teuthology.orchestra.run.smithi031.stdout:2972 read 0x4a77cb3 thru 0x4a783e8 (0x736 bytes) 2024-05-08T08:25:37.079 INFO:teuthology.orchestra.run.smithi031.stdout:2973 write 0x3ba7d54 thru 0x3baa55d (0x280a bytes) 2024-05-08T08:25:37.081 INFO:teuthology.orchestra.run.smithi031.stdout:2974 writesame 0x713deea thru 0x71481c8 data_size 0x1f(0xa2df bytes) 2024-05-08T08:25:37.084 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:37.092+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:37.086 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:25:37.092 INFO:teuthology.orchestra.run.smithi031.stdout:2975 read 0x516784f thru 0x516c118 (0x48ca bytes) 2024-05-08T08:25:37.092 INFO:teuthology.orchestra.run.smithi031.stdout:2976 read 0x276d362 thru 0x276eb5c (0x17fb bytes) 2024-05-08T08:25:37.092 INFO:teuthology.orchestra.run.smithi031.stdout:2977 punch from 0x122bab7 to 0x12367cf, (0xad18 bytes) 2024-05-08T08:25:37.093 INFO:teuthology.orchestra.run.smithi031.stdout:2978 write 0x6ad3df8 thru 0x6ae2b5d (0xed66 bytes) 2024-05-08T08:25:37.097 INFO:teuthology.orchestra.run.smithi031.stdout:2979 write 0x56d8fbd thru 0x56ddeb2 (0x4ef6 bytes) 2024-05-08T08:25:37.111 INFO:teuthology.orchestra.run.smithi031.stdout:2980 write 0xe12e39d thru 0xe134691 (0x62f5 bytes) 2024-05-08T08:25:37.114 INFO:teuthology.orchestra.run.smithi031.stdout:2981 writesame 0x8c7ac9a thru 0x8c8371e data_size 0x1(0x8a85 bytes) 2024-05-08T08:25:37.127 INFO:teuthology.orchestra.run.smithi031.stdout:2982 punch from 0x9798581 to 0x97a2994, (0xa413 bytes) 2024-05-08T08:25:37.127 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:37.128 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:2983 trunc from 0xe134692 to 0x94a90e3 2024-05-08T08:25:37.137 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:37.143 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.144 INFO:teuthology.orchestra.run.smithi031.stdout:2985 read 0x2b3c693 thru 0x2b47906 (0xb274 bytes) 2024-05-08T08:25:37.144 INFO:teuthology.orchestra.run.smithi031.stdout:2986 read 0x75fed75 thru 0x7608db8 (0xa044 bytes) 2024-05-08T08:25:37.144 INFO:teuthology.orchestra.run.smithi031.stdout:2987 writesame 0x1834c69 thru 0x183bc48 data_size 0x28(0x6fe0 bytes) 2024-05-08T08:25:37.146 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.146 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.146 INFO:teuthology.orchestra.run.smithi031.stdout:2991 write 0x52bbd9c thru 0x52bc5b1 (0x816 bytes) 2024-05-08T08:25:37.154 INFO:teuthology.orchestra.run.smithi031.stdout:2993 trunc from 0x94a90e3 to 0x58f3ee7 2024-05-08T08:25:37.164 INFO:teuthology.orchestra.run.smithi031.stdout:2995 read 0x21833c thru 0x223f4c (0xbc11 bytes) 2024-05-08T08:25:37.179 INFO:teuthology.orchestra.run.smithi031.stdout:2996 write 0xc591ada thru 0xc59e840 (0xcd67 bytes) 2024-05-08T08:25:37.183 INFO:teuthology.orchestra.run.smithi031.stdout:2997 read 0x40f6f35 thru 0x41056fa (0xe7c6 bytes) 2024-05-08T08:25:37.183 INFO:teuthology.orchestra.run.smithi031.stdout:2998 read 0x61328cc thru 0x6141d0f (0xf444 bytes) 2024-05-08T08:25:37.183 INFO:teuthology.orchestra.run.smithi031.stdout:2999 write 0x5bd69b5 thru 0x5bdb487 (0x4ad3 bytes) 2024-05-08T08:25:37.185 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:37.193+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:37.185 INFO:teuthology.orchestra.run.smithi031.stdout:3002 write 0x87dfb12 thru 0x87e21ad (0x269c bytes) 2024-05-08T08:25:37.189 INFO:teuthology.orchestra.run.smithi031.stdout:3003 write 0xb051708 thru 0xb053863 (0x215c bytes) 2024-05-08T08:25:37.192 INFO:teuthology.orchestra.run.smithi031.stdout:3004 write 0x7fe610d thru 0x7fe9954 (0x3848 bytes) 2024-05-08T08:25:37.205 INFO:teuthology.orchestra.run.smithi031.stdout:3005 trunc from 0xc59e841 to 0x3770b1e 2024-05-08T08:25:37.225 INFO:teuthology.orchestra.run.smithi031.stdout:3007 write 0x594c36e thru 0x595a300 (0xdf93 bytes) 2024-05-08T08:25:37.235 INFO:teuthology.orchestra.run.smithi031.stdout:3008 trunc from 0x595a301 to 0x9e0fcf9 2024-05-08T08:25:37.242 INFO:teuthology.orchestra.run.smithi031.stdout:3011 read 0x9788084 thru 0x978af56 (0x2ed3 bytes) 2024-05-08T08:25:37.242 INFO:teuthology.orchestra.run.smithi031.stdout:3012 read 0x769c02e thru 0x76aab56 (0xeb29 bytes) 2024-05-08T08:25:37.242 INFO:teuthology.orchestra.run.smithi031.stdout:3013 write 0x566bba5 thru 0x5679f58 (0xe3b4 bytes) 2024-05-08T08:25:37.251 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:37.251 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:37.254 INFO:teuthology.orchestra.run.smithi031.stdout:3015 writesame 0x9f620e9 thru 0x9f6630c data_size 0x1(0x4224 bytes) 2024-05-08T08:25:37.254 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.254 INFO:teuthology.orchestra.run.smithi031.stdout:3017 punch from 0x65cd1ae to 0x65ce619, (0x146b bytes) 2024-05-08T08:25:37.254 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.260 INFO:teuthology.orchestra.run.smithi031.stdout:3021 write 0xad4993e thru 0xad5934d (0xfa10 bytes) 2024-05-08T08:25:37.261 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:37.266 INFO:teuthology.orchestra.run.smithi031.stdout:3023 writesame 0x61f3276 thru 0x61f5aa5 data_size 0x8(0x2830 bytes) 2024-05-08T08:25:37.269 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.271 INFO:teuthology.orchestra.run.smithi031.stdout:3025 trunc from 0xad5934e to 0xc467301 2024-05-08T08:25:37.276 INFO:teuthology.orchestra.run.smithi031.stdout:3026 punch from 0x554eef2 to 0x5557de1, (0x8eef bytes) 2024-05-08T08:25:37.276 INFO:teuthology.orchestra.run.smithi031.stdout:3027 read 0x2a9ad63 thru 0x2aa9a46 (0xece4 bytes) 2024-05-08T08:25:37.276 INFO:teuthology.orchestra.run.smithi031.stdout:3029 writesame 0x1c2a789 thru 0x1c384b9 data_size 0x19(0xdd31 bytes) 2024-05-08T08:25:37.280 INFO:teuthology.orchestra.run.smithi031.stdout:3030 write 0xbf4b89e thru 0xbf50b92 (0x52f5 bytes) 2024-05-08T08:25:37.283 INFO:teuthology.orchestra.run.smithi031.stdout:3031 write 0x24aa4aa thru 0x24b1055 (0x6bac bytes) 2024-05-08T08:25:37.290 INFO:teuthology.orchestra.run.smithi031.stdout:3033 trunc from 0xc467301 to 0x935ef2a 2024-05-08T08:25:37.302 INFO:teuthology.orchestra.run.smithi031.stdout:3034 read 0x87720bc thru 0x877611b (0x4060 bytes) 2024-05-08T08:25:37.302 INFO:teuthology.orchestra.run.smithi031.stdout:3036 write 0x17ddb19 thru 0x17e98ad (0xbd95 bytes) 2024-05-08T08:25:37.303 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:37.310+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:37.306 INFO:teuthology.orchestra.run.smithi031.stdout:3037 read 0x38d298e thru 0x38d4b2f (0x21a2 bytes) 2024-05-08T08:25:37.317 INFO:teuthology.orchestra.run.smithi031.stdout:3038 write 0xdbb6bde thru 0xdbbef33 (0x8356 bytes) 2024-05-08T08:25:37.329 INFO:teuthology.orchestra.run.smithi031.stdout:3040 write 0xed0d134 thru 0xed1497d (0x784a bytes) 2024-05-08T08:25:37.335 INFO:teuthology.orchestra.run.smithi031.stdout:3041 read 0x51cc03c thru 0x51d5c0e (0x9bd3 bytes) 2024-05-08T08:25:37.335 INFO:teuthology.orchestra.run.smithi031.stdout:3042 read 0x8e319f4 thru 0x8e3e6fc (0xcd09 bytes) 2024-05-08T08:25:37.335 INFO:teuthology.orchestra.run.smithi031.stdout:3043 read 0x281cb3e thru 0x28238ce (0x6d91 bytes) 2024-05-08T08:25:37.335 INFO:teuthology.orchestra.run.smithi031.stdout:3045 writesame 0x98c7a57 thru 0x98d0d22 data_size 0xa(0x92cc bytes) 2024-05-08T08:25:37.339 INFO:teuthology.orchestra.run.smithi031.stdout:3046 writesame 0x288c0c3 thru 0x2897e1c data_size 0x9(0xbd5a bytes) 2024-05-08T08:25:37.351 INFO:teuthology.orchestra.run.smithi031.stdout:3047 trunc from 0xed1497e to 0x86ab913 2024-05-08T08:25:37.362 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.362 INFO:teuthology.orchestra.run.smithi031.stdout:3049 punch from 0x2e2c195 to 0x2e32b76, (0x69e1 bytes) 2024-05-08T08:25:37.372 INFO:teuthology.orchestra.run.smithi031.stdout:3050 write 0xc9f82c4 thru 0xc9fcbd8 (0x4915 bytes) 2024-05-08T08:25:37.382 INFO:teuthology.orchestra.run.smithi031.stdout:3051 trunc from 0xc9fcbd9 to 0x6dd3d06 2024-05-08T08:25:37.386 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:37.393+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:37.391 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.405 INFO:teuthology.orchestra.run.smithi031.stdout:3054 writesame 0xdb69f24 thru 0xdb6faeb data_size 0x59(0x5bc8 bytes) 2024-05-08T08:25:37.412 INFO:teuthology.orchestra.run.smithi031.stdout:3055 write 0xe894e55 thru 0xe89a86f (0x5a1b bytes) 2024-05-08T08:25:37.416 INFO:teuthology.orchestra.run.smithi031.stdout:3057 punch from 0x32381ff to 0x3243d1a, (0xbb1b bytes) 2024-05-08T08:25:37.417 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:37.417 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:37.422 INFO:teuthology.orchestra.run.smithi031.stdout:3058 trunc from 0xe89a870 to 0x8d50676 2024-05-08T08:25:37.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:37.439 INFO:teuthology.orchestra.run.smithi031.stdout:3059 trunc from 0x8d50676 to 0xe5bdf8b 2024-05-08T08:25:37.459 INFO:teuthology.orchestra.run.smithi031.stdout:3061 trunc from 0xe5bdf8b to 0x2231fa0 2024-05-08T08:25:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:3062 punch from 0x1033df4 to 0x104159a, (0xd7a6 bytes) 2024-05-08T08:25:37.470 INFO:teuthology.orchestra.run.smithi031.stdout:3063 writesame 0x15e0807 thru 0x15e5d29 data_size 0x5(0x5523 bytes) 2024-05-08T08:25:37.477 INFO:teuthology.orchestra.run.smithi031.stdout:3064 trunc from 0x2231fa0 to 0x4e1efce 2024-05-08T08:25:37.487 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:37.495+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:37.497 INFO:teuthology.orchestra.run.smithi031.stdout:3065 writesame 0xa0f9b63 thru 0xa107032 data_size 0x78(0xd4d0 bytes) 2024-05-08T08:25:37.497 INFO:teuthology.orchestra.run.smithi031.stdout:3068 writesame 0x2720bb4 thru 0x272e61c data_size 0x17(0xda69 bytes) 2024-05-08T08:25:37.500 INFO:teuthology.orchestra.run.smithi031.stdout:3070 punch from 0x346fc93 to 0x347504a, (0x53b7 bytes) 2024-05-08T08:25:37.501 INFO:teuthology.orchestra.run.smithi031.stdout:3071 punch from 0x6cdf661 to 0x6ce43f3, (0x4d92 bytes) 2024-05-08T08:25:37.512 INFO:teuthology.orchestra.run.smithi031.stdout:3072 writesame 0xeb8a4dd thru 0xeb9243c data_size 0x10(0x7f60 bytes) 2024-05-08T08:25:37.512 INFO:teuthology.orchestra.run.smithi031.stdout:3073 punch from 0x9721363 to 0x9723f95, (0x2c32 bytes) 2024-05-08T08:25:37.520 INFO:teuthology.orchestra.run.smithi031.stdout:3074 trunc from 0xeb9243d to 0x928015c 2024-05-08T08:25:37.533 INFO:teuthology.orchestra.run.smithi031.stdout:3075 write 0xb021563 thru 0xb027d9e (0x683c bytes) 2024-05-08T08:25:37.536 INFO:teuthology.orchestra.run.smithi031.stdout:3076 read 0x52f2131 thru 0x5301899 (0xf769 bytes) 2024-05-08T08:25:37.546 INFO:teuthology.orchestra.run.smithi031.stdout:3078 writesame 0xd92151e thru 0xd927d9b data_size 0x32(0x687e bytes) 2024-05-08T08:25:37.546 INFO:teuthology.orchestra.run.smithi031.stdout:3079 read 0x2579505 thru 0x2584cca (0xb7c6 bytes) 2024-05-08T08:25:37.553 INFO:teuthology.orchestra.run.smithi031.stdout:3081 write 0xe2566e3 thru 0xe25fe6e (0x978c bytes) 2024-05-08T08:25:37.556 INFO:teuthology.orchestra.run.smithi031.stdout:3082 write 0x8d2660a thru 0x8d31138 (0xab2f bytes) 2024-05-08T08:25:37.560 INFO:teuthology.orchestra.run.smithi031.stdout:3083 read 0xbac21d1 thru 0xbac7feb (0x5e1b bytes) 2024-05-08T08:25:37.560 INFO:teuthology.orchestra.run.smithi031.stdout:3084 writesame 0x655f950 thru 0x6567be5 data_size 0x5(0x8296 bytes) 2024-05-08T08:25:37.565 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.565 INFO:teuthology.orchestra.run.smithi031.stdout:3086 write 0xb012ec thru 0xb05ad7 (0x47ec bytes) 2024-05-08T08:25:37.566 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:37.566 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:37.568 INFO:teuthology.orchestra.run.smithi031.stdout:3087 write 0x243536f thru 0x2440a56 (0xb6e8 bytes) 2024-05-08T08:25:37.572 INFO:teuthology.orchestra.run.smithi031.stdout:3088 write 0x27bf4a thru 0x281fb0 (0x6067 bytes) 2024-05-08T08:25:37.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:37.577 INFO:teuthology.orchestra.run.smithi031.stdout:3089 writesame 0xa0988e6 thru 0xa09c2c9 data_size 0x5f(0x39e4 bytes) 2024-05-08T08:25:37.580 INFO:teuthology.orchestra.run.smithi031.stdout:3090 write 0x418d80d thru 0x418e54a (0xd3e bytes) 2024-05-08T08:25:37.583 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.583 INFO:teuthology.orchestra.run.smithi031.stdout:3092 write 0x652014c thru 0x6522ecc (0x2d81 bytes) 2024-05-08T08:25:37.586 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.586 INFO:teuthology.orchestra.run.smithi031.stdout:3095 write 0xa170ae3 thru 0xa1799c0 (0x8ede bytes) 2024-05-08T08:25:37.588 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:37.596+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:37.593 INFO:teuthology.orchestra.run.smithi031.stdout:3096 trunc from 0xe25fe6f to 0xa508747 2024-05-08T08:25:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:3097 punch from 0x28655cc to 0x286dcc6, (0x86fa bytes) 2024-05-08T08:25:37.612 INFO:teuthology.orchestra.run.smithi031.stdout:3098 write 0xd27c9ab thru 0xd27f729 (0x2d7f bytes) 2024-05-08T08:25:37.614 INFO:teuthology.orchestra.run.smithi031.stdout:3099 write 0xaaf0037 thru 0xaaf760e (0x75d8 bytes) 2024-05-08T08:25:37.631 INFO:teuthology.orchestra.run.smithi031.stdout:3101 trunc from 0xd27f72a to 0x172473d 2024-05-08T08:25:37.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:37.659 INFO:teuthology.orchestra.run.smithi031.stdout:3102 write 0x831a365 thru 0x8321bd6 (0x7872 bytes) 2024-05-08T08:25:37.670 INFO:teuthology.orchestra.run.smithi031.stdout:3103 trunc from 0x8321bd7 to 0xe2b6898 2024-05-08T08:25:37.675 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.676 INFO:teuthology.orchestra.run.smithi031.stdout:3106 write 0x3a46789 thru 0x3a55ab9 (0xf331 bytes) 2024-05-08T08:25:37.679 INFO:teuthology.orchestra.run.smithi031.stdout:3110 write 0x1be58db thru 0x1bf20dc (0xc802 bytes) 2024-05-08T08:25:37.685 INFO:teuthology.orchestra.run.smithi031.stdout:3111 punch from 0xbd606a9 to 0xbd64a41, (0x4398 bytes) 2024-05-08T08:25:37.685 INFO:teuthology.orchestra.run.smithi031.stdout:3112 write 0x8943dc thru 0x89b220 (0x6e45 bytes) 2024-05-08T08:25:37.689 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:37.697+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:37.690 INFO:teuthology.orchestra.run.smithi031.stdout:3113 punch from 0x11f80c8 to 0x1201a1b, (0x9953 bytes) 2024-05-08T08:25:37.690 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.690 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.696 INFO:teuthology.orchestra.run.smithi031.stdout:3116 write 0xe80341c thru 0xe812e29 (0xfa0e bytes) 2024-05-08T08:25:37.702 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.702 INFO:teuthology.orchestra.run.smithi031.stdout:3118 write 0x8589c35 thru 0x858acc4 (0x1090 bytes) 2024-05-08T08:25:37.703 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:37.703 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:37.705 INFO:teuthology.orchestra.run.smithi031.stdout:3120 write 0x9dc1e5d thru 0x9dc94fc (0x76a0 bytes) 2024-05-08T08:25:37.709 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.709 INFO:teuthology.orchestra.run.smithi031.stdout:3122 writesame 0x39df581 thru 0x39e4d64 data_size 0x14(0x57e4 bytes) 2024-05-08T08:25:37.711 INFO:teuthology.orchestra.run.smithi031.stdout:3123 punch from 0x6fa2627 to 0x6fa935a, (0x6d33 bytes) 2024-05-08T08:25:37.711 INFO:teuthology.orchestra.run.smithi031.stdout:3124 write 0x20ddcd2 thru 0x20de0ff (0x42e bytes) 2024-05-08T08:25:37.714 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:37.714 INFO:teuthology.orchestra.run.smithi031.stdout:3125 write 0x7072acc thru 0x707e40c (0xb941 bytes) 2024-05-08T08:25:37.723 INFO:teuthology.orchestra.run.smithi031.stdout:3126 trunc from 0xe812e2a to 0x94e6c3d 2024-05-08T08:25:37.743 INFO:teuthology.orchestra.run.smithi031.stdout:3127 write 0xb679dfb thru 0xb685335 (0xb53b bytes) 2024-05-08T08:25:37.748 INFO:teuthology.orchestra.run.smithi031.stdout:3128 write 0x426598f thru 0x4268665 (0x2cd7 bytes) 2024-05-08T08:25:37.752 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.752 INFO:teuthology.orchestra.run.smithi031.stdout:3131 read 0x1511c1d thru 0x1512b54 (0xf38 bytes) 2024-05-08T08:25:37.752 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.752 INFO:teuthology.orchestra.run.smithi031.stdout:3135 read 0x2f75951 thru 0x2f7d724 (0x7dd4 bytes) 2024-05-08T08:25:37.752 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:37.752 INFO:teuthology.orchestra.run.smithi031.stdout:3137 punch from 0x4e4ed6 to 0x4ea9c3, (0x5aed bytes) 2024-05-08T08:25:37.752 INFO:teuthology.orchestra.run.smithi031.stdout:3138 read 0x1ee3289 thru 0x1eef9b3 (0xc72b bytes) 2024-05-08T08:25:37.760 INFO:teuthology.orchestra.run.smithi031.stdout:3139 write 0xd84a4a2 thru 0xd852973 (0x84d2 bytes) 2024-05-08T08:25:37.764 INFO:teuthology.orchestra.run.smithi031.stdout:3140 read 0x4e3c3d3 thru 0x4e4be20 (0xfa4e bytes) 2024-05-08T08:25:37.764 INFO:teuthology.orchestra.run.smithi031.stdout:3141 clone 17 order 18 su 65536 sc 13 2024-05-08T08:25:37.790 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:37.798+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:37.798 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:37.798 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:37.798 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:37.816 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:37.816 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6562 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.907406169000001, 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024048491000000002, 2024-05-08T08:25:37.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6562 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:37.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931270+0000", 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.19e-06 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931276+0000", 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4790000000000004e-06 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931286+0000", 2024-05-08T08:25:37.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.044e-05 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931292+0000", 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931346+0000", 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4274999999999998e-05 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931398+0000", 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1999999999999997e-05 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.935504+0000", 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041055170000000004 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955221+0000", 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019717443000000001 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955234+0000", 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2191999999999999e-05 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955271+0000", 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7008000000000002e-05 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955317+0000", 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6796999999999999e-05 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7004 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7077888~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650456321, 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016625803000000001, 2024-05-08T08:25:37.837 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7004 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188222+0000", 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7219999999999999e-06 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188229+0000", 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6359999999999997e-06 2024-05-08T08:25:37.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188234+0000", 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.502e-06 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203959+0000", 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015725330999999999 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204038+0000", 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8379000000000003e-05 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204845+0000", 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080675500000000004 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7020 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650367121999999, 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017204636999999998, 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7020 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:37.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188311+0000", 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188317+0000", 2024-05-08T08:25:37.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7959999999999998e-06 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188322+0000", 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204639+0000", 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016316714 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204730+0000", 2024-05-08T08:25:37.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0771000000000002e-05 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205513+0000", 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078277700000000002 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7021 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3145728~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650346145, 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017850441000000002, 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7021 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6199999999999999e-07 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188331+0000", 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1069999999999999e-06 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188338+0000", 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188343+0000", 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205565+0000", 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017221841000000002 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205665+0000", 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9530000000000004e-05 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206179+0000", 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051477999999999995 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7022 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 5767168~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650325025000001, 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018456931999999999, 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7022 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188352+0000", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188358+0000", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188363+0000", 2024-05-08T08:25:37.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206229+0000", 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017865941 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206327+0000", 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7888999999999997e-05 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206807+0000", 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048019000000000002 2024-05-08T08:25:37.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7038 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650219120999999, 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019193815999999999, 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7038 2024-05-08T08:25:37.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188457+0000", 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188461+0000", 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188469+0000", 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188477+0000", 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3559999999999993e-06 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206829+0000", 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018352456 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206928+0000", 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8981999999999995e-05 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207650+0000", 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072148600000000002 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7039 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650186380000001, 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491153000000001, 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7039 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188492+0000", 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188499+0000", 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7469999999999999e-06 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188505+0000", 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207495+0000", 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990683000000001 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207577+0000", 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.174e-05 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207980+0000", 2024-05-08T08:25:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040296899999999998 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7040 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650163091, 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020064733000000001, 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7040 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188516+0000", 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999999e-06 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188522+0000", 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188527+0000", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208019+0000", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491871000000001 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208118+0000", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8855000000000001e-05 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208577+0000", 2024-05-08T08:25:37.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000458784 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7056 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650082267999998, 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020583127999999999, 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.865 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-05-08T08:25:37.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188595+0000", 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0679999999999999e-06 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188603+0000", 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999997e-06 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188609+0000", 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208623+0000", 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0200141 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208713+0000", 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9806e-05 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209176+0000", 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046324300000000001 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7057 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4718592~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.650029150999998, 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021149303000000001, 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188647+0000", 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188650+0000", 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7100000000000001e-06 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188657+0000", 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188665+0000", 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4410000000000004e-06 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209211+0000", 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020546426999999999 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209299+0000", 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8139000000000004e-05 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209795+0000", 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049584800000000001 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.872 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7058 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7340032~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.649998837999998, 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021706842, 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7058 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:37.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188677+0000", 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188681+0000", 2024-05-08T08:25:37.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188688+0000", 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.494e-06 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188693+0000", 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209838+0000", 2024-05-08T08:25:37.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021144677000000001 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209901+0000", 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3322000000000001e-05 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210383+0000", 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048203800000000001 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.876 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7073 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2359296~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.649903329000001, 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022130316000000001, 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188775+0000", 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188782+0000", 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8630000000000008e-06 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188787+0000", 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210421+0000", 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021633454999999999 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210481+0000", 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9911000000000003e-05 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210902+0000", 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042136000000000002 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7074 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4980736~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.649881033, 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022761245999999999, 2024-05-08T08:25:37.880 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188795+0000", 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188797+0000", 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188803+0000", 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188807+0000", 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210951+0000", 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022143843 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211043+0000", 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2044000000000002e-05 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211555+0000", 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051209900000000002 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7075 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7602176~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.649861105999999, 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025123269, 2024-05-08T08:25:37.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7075 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188817+0000", 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188823+0000", 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188828+0000", 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211585+0000", 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022757553999999999 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211640+0000", 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000002e-05 2024-05-08T08:25:37.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213937+0000", 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002297182 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7090 3.12 3:4c3c5acc:::rbd_data.119120fdbc10.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1572864~19660800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 21.640386981999999, 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017096892999999998, 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:37.888 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7090 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198290+0000", 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.666e-06 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198297+0000", 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198308+0000", 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0641e-05 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198314+0000", 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7850000000000003e-06 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213971+0000", 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015656415 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214141+0000", 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016982999999999999 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214349+0000", 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020874199999999999 2024-05-08T08:25:37.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215032+0000", 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068226599999999995 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215223+0000", 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191263 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215286+0000", 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000004e-05 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215385+0000", 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9161999999999997e-05 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10928 3.6 3:66234790:::rbd_data.11917ede465c.0000000000000000:9 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.699677205, 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016562112, 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.893 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10928 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139012+0000", 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3908e-05 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139020+0000", 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.96e-06 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139026+0000", 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5219999999999998e-06 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155334+0000", 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016308699999999999 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155417+0000", 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2229000000000002e-05 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155560+0000", 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014340300000000001 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10943 3.6 3:62dc9057:::rbd_data.119177d1b31a.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.697915219, 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018993326000000001, 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.897 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10943 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140761+0000", 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999999e-07 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140765+0000", 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140775+0000", 2024-05-08T08:25:37.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9690000000000006e-06 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140782+0000", 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156122+0000", 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015340006999999999 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156209+0000", 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7249000000000004e-05 2024-05-08T08:25:37.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159642+0000", 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034333430000000002 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159700+0000", 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8078999999999997e-05 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159753+0000", 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3019000000000002e-05 2024-05-08T08:25:37.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:13022 3.17 3:ee16c3de:::rbd_data.1191c87c6f01.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 20828828~4380244] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.304740762, 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020689712999999998, 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.902 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13022 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533938+0000", 2024-05-08T08:25:37.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9879999999999998e-06 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534933+0000", 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099514400000000011 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534955+0000", 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1225000000000001e-05 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534965+0000", 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597999999999999e-05 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535030+0000", 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5068999999999997e-05 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535157+0000", 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126501 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535401+0000", 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024435700000000001 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.540584+0000", 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051824109999999996 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554524+0000", 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013939890999999999 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554584+0000", 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554624+0000", 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0028999999999999e-05 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:23289 3.3 e41/18 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head v 41'115, mlcod=39'113)", 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.3149859829999997, 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024290019999999999, 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523690+0000", 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2900000000000002e-07 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545794+0000", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022104220000000001 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545810+0000", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5420000000000001e-05 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545818+0000", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0979999999999994e-06 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545894+0000", 2024-05-08T08:25:37.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6579e-05 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545914+0000", 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9372e-05 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547942+0000", 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020284220000000002 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547979+0000", 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6980000000000002e-05 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:25389 3.17 3:ea3e87a1:::rbd_data.11911e381319.0000000000000114:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 2.890682108, 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020343165999999999, 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:37.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 25389 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947994+0000", 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1319999999999999e-06 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947998+0000", 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2830000000000003e-06 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948009+0000", 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.058e-05 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948014+0000", 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948057+0000", 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2330000000000003e-05 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948137+0000", 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9985000000000001e-05 2024-05-08T08:25:37.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.952162+0000", 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040247959999999998 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968241+0000", 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016079751 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968253+0000", 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1296e-05 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968290+0000", 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7011000000000002e-05 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968336+0000", 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6612e-05 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:37.916 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:37.917 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:37.917 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:37.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:37.918 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:37.919 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:37.919 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:37.920 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:37.920 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:37.920 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:37.921 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:37.921 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:37.925+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:38.018 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:38.026+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:38.068 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:38.068 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:38.069 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:38.069 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:38.076 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-05-08T08:25:38.076 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-05-08T08:25:38.077 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T08:25:38.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T08:25:38.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:38.119 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:38.127+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:38.119 INFO:tasks.ceph.osd.1:Started 2024-05-08T08:25:38.120 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:38.173 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone16 from 0xd852974 (overlap 0x4e1ca6) to 0x36ada7 2024-05-08T08:25:38.220 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:38.227+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:25:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:25:38.237 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:config diff 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:config diff get 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:config get 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:config help [] 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:config set ... 2024-05-08T08:25:38.238 INFO:teuthology.orchestra.run.smithi031.stderr:admin_socket: invalid command 2024-05-08T08:25:38.240 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:25:38.241 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-05-08T08:25:38.245 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:38.245 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:38.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:38.267 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-05-08T08:25:38.273 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.273 INFO:teuthology.orchestra.run.smithi031.stdout:3143 read 0xb44ca9f thru 0xb452724 (0x5c86 bytes) 2024-05-08T08:25:38.273 INFO:teuthology.orchestra.run.smithi031.stdout:3144 write 0x1fc86f1 thru 0x1fd5fbd (0xd8cd bytes) 2024-05-08T08:25:38.281 INFO:teuthology.orchestra.run.smithi031.stdout:3145 read 0x2b19f01 thru 0x2b273f0 (0xd4f0 bytes) 2024-05-08T08:25:38.281 INFO:teuthology.orchestra.run.smithi031.stdout:3146 writesame 0x8cd5f39 thru 0x8ce3691 data_size 0x1(0xd759 bytes) 2024-05-08T08:25:38.303 INFO:teuthology.orchestra.run.smithi031.stdout:3147 read 0x611e6d2 thru 0x6124bab (0x64da bytes) 2024-05-08T08:25:38.303 INFO:teuthology.orchestra.run.smithi031.stdout:3149 punch from 0x7a806c2 to 0x7a90000, (0xf93e bytes) 2024-05-08T08:25:38.309 INFO:teuthology.orchestra.run.smithi031.stdout:3150 trunc from 0xd852974 to 0x80a8700 2024-05-08T08:25:38.320 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:38.327+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:38.335 INFO:teuthology.orchestra.run.smithi031.stdout:3151 punch from 0x1f032fa to 0x1f129a2, (0xf6a8 bytes) 2024-05-08T08:25:38.335 INFO:teuthology.orchestra.run.smithi031.stdout:3152 write 0x5853666 thru 0x585b9d4 (0x836f bytes) 2024-05-08T08:25:38.337 INFO:teuthology.orchestra.run.smithi031.stdout:3153 write 0x4768c2a thru 0x4769780 (0xb57 bytes) 2024-05-08T08:25:38.341 INFO:teuthology.orchestra.run.smithi031.stdout:3154 read 0x74659cb thru 0x7469c8d (0x42c3 bytes) 2024-05-08T08:25:38.341 INFO:teuthology.orchestra.run.smithi031.stdout:3155 write 0x6345db4 thru 0x6351371 (0xb5be bytes) 2024-05-08T08:25:38.344 INFO:teuthology.orchestra.run.smithi031.stdout:3156 writesame 0x2b86a4b thru 0x2b8ed3f data_size 0x4b(0x82f5 bytes) 2024-05-08T08:25:38.360 INFO:teuthology.orchestra.run.smithi031.stdout:3157 write 0xe57bb65 thru 0xe58a2db (0xe777 bytes) 2024-05-08T08:25:38.363 INFO:teuthology.orchestra.run.smithi031.stdout:3159 write 0x501d0bf thru 0x5023c4a (0x6b8c bytes) 2024-05-08T08:25:38.371 INFO:teuthology.orchestra.run.smithi031.stdout:3160 trunc from 0xe58a2dc to 0xb9cb112 2024-05-08T08:25:38.387 INFO:teuthology.orchestra.run.smithi031.stdout:3162 trunc from 0xb9cb112 to 0x626bc5b 2024-05-08T08:25:38.394 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2370 3.4 e24/18 3:237be9c1:::rbd_data.119116cc7dd0.00000000000001eb:head v 24'46, mlcod=23'42)", 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.679974452, 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029578109999999999, 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721287+0000", 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7199999999999997e-07 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721294+0000", 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9800000000000001e-06 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721309+0000", 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5116999999999999e-05 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9030000000000004e-06 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721383+0000", 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6111000000000005e-05 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721406+0000", 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2453e-05 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724192+0000", 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027864909999999999 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724244+0000", 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.675761144999999, 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.406067608000001, 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:25:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:25:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.376979842000001, 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:25:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:25:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:25:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.261828145999999, 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:25:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:25:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:25:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1381817240000007, 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:25:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:25:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:25:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1381105770000008, 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:25:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1377648469999997, 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:25:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:25:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1377235579999994, 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:25:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20624 3.4 3:21e3449a:::rbd_data.119174045df.0000000000000045:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:38.435 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.1374252390000006, 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029912699999999999, 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0200000000000001e-07 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263841+0000", 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263854+0000", 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2937e-05 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263862+0000", 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6729999999999992e-06 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263949+0000", 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7406999999999998e-05 2024-05-08T08:25:38.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263954+0000", 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266741+0000", 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027867640000000002 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266827+0000", 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5780999999999997e-05 2024-05-08T08:25:38.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20847 3.0 3:04dbc92b:::rbd_data.1191948f0501.0000000000000008:e [read 487424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9291401199999996, 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029128679999999999, 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.440 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20847 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472121+0000", 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6399999999999998e-07 2024-05-08T08:25:38.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472123+0000", 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.841e-06 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472128+0000", 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4280000000000004e-06 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.472133+0000", 2024-05-08T08:25:38.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3150000000000002e-06 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.474903+0000", 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027698620000000001 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.474958+0000", 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5067e-05 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.475034+0000", 2024-05-08T08:25:38.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5990999999999998e-05 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21116 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 339968~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477570+0000", 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.923690745, 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027748040000000001, 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.444 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21116 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477570+0000", 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477570+0000", 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477572+0000", 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.809e-06 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477579+0000", 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7020000000000004e-06 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477584+0000", 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0590000000000002e-06 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477588+0000", 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6149999999999999e-06 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480189+0000", 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002600158 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480243+0000", 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4629999999999997e-05 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.447 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480345+0000", 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010183099999999999 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21117 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 413696~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9236663089999997, 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003161652, 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.448 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21117 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477596+0000", 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8050000000000001e-06 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477599+0000", 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.616e-06 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477604+0000", 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477608+0000", 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480366+0000", 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027578020000000002 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480420+0000", 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3735e-05 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480756+0000", 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033637199999999998 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.9236434510000002, 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:25:38.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21737 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 376832~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624108+0000", 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.7771529019999992, 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0028862470000000002, 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21737 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624108+0000", 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624108+0000", 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624109+0000", 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4799999999999998e-07 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624111+0000", 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.553e-06 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624118+0000", 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9920000000000001e-06 2024-05-08T08:25:38.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624123+0000", 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2900000000000002e-06 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.626801+0000", 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002677184 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.626870+0000", 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9692999999999998e-05 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.459 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.626994+0000", 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012398699999999999 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21738 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 450560~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.7771302939999991, 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031591710000000001, 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.460 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21738 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624133+0000", 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624140+0000", 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624145+0000", 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627084+0000", 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029395129999999999 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627161+0000", 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7096999999999997e-05 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627290+0000", 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012866399999999999 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.6314296210000006, 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.464 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.465 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:25:38.466 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:25:38.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:38.468 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.9042275850000001, 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.471 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:25:38.472 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.6852472199999999, 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.473 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.474 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:25:38.475 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.476 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.197021855, 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:25:38.477 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.478 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:25:38.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.481 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:38.482 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:38.483 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.483 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.483 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.483 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.483 INFO:teuthology.orchestra.run.smithi031.stdout:3168 writesame 0x234672a thru 0x234ce25 data_size 0x34(0x66fc bytes) 2024-05-08T08:25:38.483 INFO:teuthology.orchestra.run.smithi031.stdout:3169 write 0x5c1ab88 thru 0x5c25aa4 (0xaf1d bytes) 2024-05-08T08:25:38.483 INFO:teuthology.orchestra.run.smithi031.stdout:3170 read 0x1aec4bf thru 0x1afb624 (0xf166 bytes) 2024-05-08T08:25:38.484 INFO:teuthology.orchestra.run.smithi031.stdout:3171 write 0x6f46ebd thru 0x6f4f2ad (0x83f1 bytes) 2024-05-08T08:25:38.484 INFO:teuthology.orchestra.run.smithi031.stdout:3172 write 0x5399f99 thru 0x539bb2e (0x1b96 bytes) 2024-05-08T08:25:38.484 INFO:teuthology.orchestra.run.smithi031.stdout:3173 trunc from 0x6f4f2ae to 0x2234d2e 2024-05-08T08:25:38.484 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:38.490+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:38.494 INFO:teuthology.orchestra.run.smithi031.stdout:3174 writesame 0xb9e59eb thru 0xb9ec3a1 data_size 0x5d(0x69b7 bytes) 2024-05-08T08:25:38.494 INFO:teuthology.orchestra.run.smithi031.stdout:3176 read 0x9e2a5fe thru 0x9e2e43f (0x3e42 bytes) 2024-05-08T08:25:38.494 INFO:teuthology.orchestra.run.smithi031.stdout:3178 read 0xada891b thru 0xadb5d9c (0xd482 bytes) 2024-05-08T08:25:38.494 INFO:teuthology.orchestra.run.smithi031.stdout:3181 write 0x44071a3 thru 0x440ad09 (0x3b67 bytes) 2024-05-08T08:25:38.496 INFO:teuthology.orchestra.run.smithi031.stdout:3182 punch from 0x9cc6e7e to 0x9cc9110, (0x2292 bytes) 2024-05-08T08:25:38.496 INFO:teuthology.orchestra.run.smithi031.stdout:3183 write 0xaa3ed2e thru 0xaa49f9b (0xb26e bytes) 2024-05-08T08:25:38.511 INFO:teuthology.orchestra.run.smithi031.stdout:3184 write 0xbaf2c19 thru 0xbafeaf8 (0xbee0 bytes) 2024-05-08T08:25:38.514 INFO:teuthology.orchestra.run.smithi031.stdout:3185 write 0x213a8af thru 0x21422f9 (0x7a4b bytes) 2024-05-08T08:25:38.515 INFO:teuthology.orchestra.run.smithi031.stdout:3186 write 0xb8b5c7a thru 0xb8bdd55 (0x80dc bytes) 2024-05-08T08:25:38.521 INFO:teuthology.orchestra.run.smithi031.stdout:3188 trunc from 0xbafeaf9 to 0xe3d86ab 2024-05-08T08:25:38.526 INFO:teuthology.orchestra.run.smithi031.stdout:3189 write 0x71d5216 thru 0x71de5da (0x93c5 bytes) 2024-05-08T08:25:38.529 INFO:teuthology.orchestra.run.smithi031.stdout:3191 write 0x8f7f147 thru 0x8f8c2bd (0xd177 bytes) 2024-05-08T08:25:38.533 INFO:teuthology.orchestra.run.smithi031.stdout:3194 write 0xa1d4f0 thru 0xa24e85 (0x7996 bytes) 2024-05-08T08:25:38.537 INFO:teuthology.orchestra.run.smithi031.stdout:3195 read 0xe38b9c3 thru 0xe395d15 (0xa353 bytes) 2024-05-08T08:25:38.537 INFO:teuthology.orchestra.run.smithi031.stdout:3196 write 0x2129697 thru 0x212fa4e (0x63b8 bytes) 2024-05-08T08:25:38.538 INFO:teuthology.orchestra.run.smithi031.stdout:3197 write 0xa225b59 thru 0xa23003d (0xa4e5 bytes) 2024-05-08T08:25:38.541 INFO:teuthology.orchestra.run.smithi031.stdout:3198 writesame 0xdc55e13 thru 0xdc58dd8 data_size 0x5(0x2fc6 bytes) 2024-05-08T08:25:38.545 INFO:teuthology.orchestra.run.smithi031.stdout:3199 read 0x817e7aa thru 0x8184589 (0x5de0 bytes) 2024-05-08T08:25:38.545 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.559 INFO:teuthology.orchestra.run.smithi031.stdout:3201 trunc from 0xe3d86ab to 0x15339ff 2024-05-08T08:25:38.583 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:38.590+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:38.609 INFO:teuthology.orchestra.run.smithi031.stdout:3203 writesame 0xdefe3d9 thru 0xdefffb9 data_size 0xd(0x1be1 bytes) 2024-05-08T08:25:38.609 INFO:teuthology.orchestra.run.smithi031.stdout:3204 read 0xb21e97f thru 0xb22069b (0x1d1d bytes) 2024-05-08T08:25:38.610 INFO:teuthology.orchestra.run.smithi031.stdout:3205 write 0x3e01e4e thru 0x3e0a866 (0x8a19 bytes) 2024-05-08T08:25:38.624 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:38.624 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:38.624 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:38.624 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:38.625 INFO:teuthology.orchestra.run.smithi031.stdout:3206 trunc from 0xdefffba to 0x212e2fd 2024-05-08T08:25:38.634 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:38.635 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.635 INFO:teuthology.orchestra.run.smithi031.stdout:3210 read 0x1a604ed thru 0x1a6ba93 (0xb5a7 bytes) 2024-05-08T08:25:38.649 INFO:teuthology.orchestra.run.smithi031.stdout:3211 write 0x917b164 thru 0x918af16 (0xfdb3 bytes) 2024-05-08T08:25:38.662 INFO:teuthology.orchestra.run.smithi031.stdout:3212 writesame 0xbc0431e thru 0xbc06dab data_size 0xd(0x2a8e bytes) 2024-05-08T08:25:38.671 INFO:teuthology.orchestra.run.smithi031.stdout:3214 writesame 0xeaf716d thru 0xeafb335 data_size 0x1(0x41c9 bytes) 2024-05-08T08:25:38.671 INFO:teuthology.orchestra.run.smithi031.stdout:3215 read 0xb0a51a9 thru 0xb0b07b7 (0xb60f bytes) 2024-05-08T08:25:38.671 INFO:teuthology.orchestra.run.smithi031.stdout:3216 read 0xcf8cca7 thru 0xcf971ec (0xa546 bytes) 2024-05-08T08:25:38.672 INFO:teuthology.orchestra.run.smithi031.stdout:3218 write 0x9fc74ac thru 0x9fd64ee (0xf043 bytes) 2024-05-08T08:25:38.677 INFO:teuthology.orchestra.run.smithi031.stdout:3219 write 0x68b6b25 thru 0x68c0092 (0x956e bytes) 2024-05-08T08:25:38.682 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:3222 punch from 0x60d20f7 to 0x60df94e, (0xd857 bytes) 2024-05-08T08:25:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:3224 punch from 0xade3a72 to 0xaded796, (0x9d24 bytes) 2024-05-08T08:25:38.683 INFO:teuthology.orchestra.run.smithi031.stdout:3225 write 0x2193348 thru 0x21a2e48 (0xfb01 bytes) 2024-05-08T08:25:38.683 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:38.691+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:38.686 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.686 INFO:teuthology.orchestra.run.smithi031.stdout:3228 write 0x16bf9e7 thru 0x16cd1ed (0xd807 bytes) 2024-05-08T08:25:38.689 INFO:teuthology.orchestra.run.smithi031.stdout:3230 read 0x5aeb089 thru 0x5af6b19 (0xba91 bytes) 2024-05-08T08:25:38.689 INFO:teuthology.orchestra.run.smithi031.stdout:3231 read 0x2025686 thru 0x202bbe1 (0x655c bytes) 2024-05-08T08:25:38.689 INFO:teuthology.orchestra.run.smithi031.stdout:3232 write 0x3a2be9e thru 0x3a30b53 (0x4cb6 bytes) 2024-05-08T08:25:38.703 INFO:teuthology.orchestra.run.smithi031.stdout:3233 trunc from 0xeafb336 to 0x5bf3c24 2024-05-08T08:25:38.716 INFO:teuthology.orchestra.run.smithi031.stdout:3234 trunc from 0x5bf3c24 to 0x93778b4 2024-05-08T08:25:38.722 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.722 INFO:teuthology.orchestra.run.smithi031.stdout:3236 read 0x282cfcf thru 0x28313c2 (0x43f4 bytes) 2024-05-08T08:25:38.723 INFO:teuthology.orchestra.run.smithi031.stdout:3237 writesame 0x8d21657 thru 0x8d244fe data_size 0x4(0x2ea8 bytes) 2024-05-08T08:25:38.726 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:38.733+0000 7f0060411640 -1 Falling back to public interface 2024-05-08T08:25:38.729 INFO:teuthology.orchestra.run.smithi031.stdout:3238 trunc from 0x93778b4 to 0x621b7f0 2024-05-08T08:25:38.741 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.741 INFO:teuthology.orchestra.run.smithi031.stdout:3240 punch from 0x45fe4cb to 0x4601fc5, (0x3afa bytes) 2024-05-08T08:25:38.741 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:38.741 INFO:teuthology.orchestra.run.smithi031.stdout:3242 read 0x36e6e24 thru 0x36e8ec4 (0x20a1 bytes) 2024-05-08T08:25:38.741 INFO:teuthology.orchestra.run.smithi031.stdout:3243 clone 18 order 23 su 131072 sc 2 2024-05-08T08:25:38.758 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:38.759 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:38.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:38.808 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:38.816+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:38.885 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:38.893+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6425 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:head [set-alloc-hint object_size 8388608 write_size 8388608,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:38.919 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.217225285000001, 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03287702, 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6425 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708823+0000", 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708825+0000", 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7e-06 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708833+0000", 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1349999999999992e-06 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708851+0000", 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7961e-05 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708859+0000", 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2439999999999993e-06 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.708920+0000", 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1178000000000003e-05 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.709031+0000", 2024-05-08T08:25:38.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011102 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.714353+0000", 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0053222440000000003 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741561+0000", 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.027207346 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741581+0000", 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0854000000000001e-05 2024-05-08T08:25:38.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741644+0000", 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2526000000000004e-05 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.741700+0000", 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5812e-05 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6447 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:38.924 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.151791535000001, 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020412153999999998, 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6447 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774257+0000", 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774258+0000", 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.731e-06 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774267+0000", 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8869999999999998e-06 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774285+0000", 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7399000000000001e-05 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774293+0000", 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1089999999999998e-06 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774350+0000", 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.774416+0000", 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6166000000000003e-05 2024-05-08T08:25:38.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.777145+0000", 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027283160000000002 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794536+0000", 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017391163000000001 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794555+0000", 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9377000000000001e-05 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794614+0000", 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8829000000000002e-05 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.794669+0000", 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4537999999999999e-05 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7046 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 5505024~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.737145810000001, 2024-05-08T08:25:38.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020079657000000001, 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7046 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188902+0000", 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188903+0000", 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188905+0000", 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2450000000000001e-06 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188911+0000", 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-06 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188915+0000", 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.515e-06 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208342+0000", 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019426335999999999 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208397+0000", 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5417000000000002e-05 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208982+0000", 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058485500000000001 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7047 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 8126464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.737126932999999, 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020213064999999999, 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7047 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:38.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188921+0000", 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2099999999999998e-07 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188923+0000", 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7850000000000001e-06 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188929+0000", 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188934+0000", 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7360000000000001e-06 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209012+0000", 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020078822999999999 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209078+0000", 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5859999999999996e-05 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209134+0000", 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5952999999999998e-05 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7051 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 1835008~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.737107902999998, 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020448772, 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7051 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188940+0000", 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188941+0000", 2024-05-08T08:25:38.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2800000000000003e-07 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188943+0000", 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3020000000000002e-06 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188949+0000", 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7819999999999999e-06 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188954+0000", 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.164e-06 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209169+0000", 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020215151000000001 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209252+0000", 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3341e-05 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209389+0000", 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000136704 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7052 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 4456448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.737088745000001, 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020652770000000001, 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7052 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188959+0000", 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188960+0000", 2024-05-08T08:25:38.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7999999999999996e-07 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188966+0000", 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5370000000000003e-06 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188972+0000", 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9660000000000001e-06 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188977+0000", 2024-05-08T08:25:38.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3939999999999998e-06 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209425+0000", 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020447775000000001 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209511+0000", 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6273999999999994e-05 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209612+0000", 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101344 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7062 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 1572864~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736910284, 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021205383000000001, 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7062 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189138+0000", 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7800000000000002e-07 2024-05-08T08:25:38.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189140+0000", 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189146+0000", 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.716e-06 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189151+0000", 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5820000000000003e-06 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209642+0000", 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020491067000000002 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209730+0000", 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8078000000000006e-05 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210343+0000", 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00061346199999999995 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7063 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 4194304~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736891740000001, 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022006687, 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7063 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189156+0000", 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189157+0000", 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1600000000000002e-07 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189159+0000", 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9570000000000001e-06 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189164+0000", 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189169+0000", 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3680000000000004e-06 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210391+0000", 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021222382000000001 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210478+0000", 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7168000000000003e-05 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211163+0000", 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000684693 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7064 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736873590999998, 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022198532, 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7064 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:38.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189175+0000", 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1300000000000001e-07 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189177+0000", 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.88e-06 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189182+0000", 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7910000000000001e-06 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189187+0000", 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4499999999999997e-06 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211198+0000", 2024-05-08T08:25:38.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022011134000000002 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211289+0000", 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0537000000000004e-05 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211373+0000", 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4426999999999997e-05 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7068 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 524288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.956 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736855479999999, 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022446079000000001, 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7068 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189193+0000", 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8599999999999998e-07 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189195+0000", 2024-05-08T08:25:38.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9039999999999999e-06 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189201+0000", 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1109999999999996e-06 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189216+0000", 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4758e-05 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211430+0000", 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022213944999999999 2024-05-08T08:25:38.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211529+0000", 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9504000000000001e-05 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211639+0000", 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010937099999999999 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7069 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 3145728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.960 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736824500000001, 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022643852999999999, 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7069 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189224+0000", 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189227+0000", 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6410000000000002e-06 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189234+0000", 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8499999999999996e-06 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189244+0000", 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597e-05 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211685+0000", 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022441040999999998 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211778+0000", 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2449999999999997e-05 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211867+0000", 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9833999999999994e-05 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7077 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736714503000002, 2024-05-08T08:25:38.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019800081000000001, 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7077 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189334+0000", 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189336+0000", 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189342+0000", 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0220000000000001e-06 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189347+0000", 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5299999999999998e-06 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208501+0000", 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019154612000000001 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208603+0000", 2024-05-08T08:25:38.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101408 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209134+0000", 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000530938 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7078 3.1e 3:7afff813:::rbd_data.119152d8614e.0000000000000002:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736695116, 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020694535, 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.968 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7078 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189353+0000", 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189355+0000", 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0779999999999998e-06 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189361+0000", 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.694e-06 2024-05-08T08:25:38.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189366+0000", 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.42e-06 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209163+0000", 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01979707 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209236+0000", 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3203999999999999e-05 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210048+0000", 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000811683 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7079 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 1835008~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736676936999999, 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024715221999999998, 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.972 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7079 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189371+0000", 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189372+0000", 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5699999999999998e-07 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189374+0000", 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.052e-06 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189379+0000", 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8950000000000003e-06 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189384+0000", 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6959999999999996e-06 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211890+0000", 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022506291000000001 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211938+0000", 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7750999999999997e-05 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214086+0000", 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00214818 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7080 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 4456448~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736657629, 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025413182999999999, 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7080 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189390+0000", 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189391+0000", 2024-05-08T08:25:38.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4799999999999999e-07 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189394+0000", 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0230000000000001e-06 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189402+0000", 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3780000000000001e-06 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189414+0000", 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1639e-05 2024-05-08T08:25:38.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214132+0000", 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024717807000000001 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214218+0000", 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6093000000000004e-05 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214804+0000", 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00058589500000000001 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7081 3.7 3:e1401663:::rbd_data.119152d8614e.0000000000000003:7 [sparse-read 7077888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736615366999999, 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025598254000000001, 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7081 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189433+0000", 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.03e-07 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189436+0000", 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189443+0000", 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189448+0000", 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0810000000000001e-06 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214836+0000", 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025387742000000001 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214926+0000", 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0665e-05 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215031+0000", 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104678 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7085 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 786432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736592228999999, 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025822804000000001, 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7085 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189456+0000", 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9099999999999999e-07 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189459+0000", 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3379999999999999e-06 2024-05-08T08:25:38.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189465+0000", 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.088e-06 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189470+0000", 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8640000000000004e-06 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215073+0000", 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025603819999999999 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215180+0000", 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106215 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215279+0000", 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9087999999999997e-05 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7086 3.7 3:e2a0a916:::rbd_data.119152d8614e.0000000000000016:7 [sparse-read 3407872~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:38.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.736572387999999, 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.026065310000000001, 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7086 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189476+0000", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189478+0000", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0590000000000001e-06 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189486+0000", 2024-05-08T08:25:38.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.841e-06 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.189491+0000", 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0679999999999996e-06 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215315+0000", 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025824427 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215411+0000", 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5606000000000006e-05 2024-05-08T08:25:38.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215541+0000", 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012996899999999999 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:21944 3.c e40/18 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head v 40'113, mlcod=40'111)", 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.689252+0000", 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.2367956370000002, 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.023217741, 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.689252+0000", 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.689252+0000", 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.689255+0000", 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.193e-06 2024-05-08T08:25:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.703132+0000", 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013877344 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.703151+0000", 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8566000000000001e-05 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.703159+0000", 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7600000000000008e-06 2024-05-08T08:25:38.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.703238+0000", 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9122000000000006e-05 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.703258+0000", 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9329000000000001e-05 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.712429+0000", 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091716230000000003 2024-05-08T08:25:38.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.712470+0000", 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0803999999999998e-05 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23532 3.2 3:47cdf4e4:::rbd_data.1191ea0f1b1f.000000000000000c:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.828809+0000", 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.0972393520000008, 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020069296, 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:38.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23532 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828809+0000", 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828809+0000", 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828810+0000", 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1939999999999999e-06 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828814+0000", 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3529999999999997e-06 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828825+0000", 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0827e-05 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828831+0000", 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4070000000000001e-06 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828889+0000", 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8205000000000003e-05 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.828941+0000", 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2481000000000001e-05 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:38.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.832643+0000", 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037018519999999998 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.848805+0000", 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016162329 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.848817+0000", 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.128e-05 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.848847+0000", 2024-05-08T08:25:38.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0621999999999999e-05 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.848878+0000", 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0746e-05 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:39.000 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:39.001 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:39.002 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:39.009+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:39.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:25:39.101 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:39.109+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:39.110 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:39.111 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:39.111 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:39.120 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:39.177 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone17 from 0x621b7f0 (overlap 0x15339ff) to 0x0 2024-05-08T08:25:39.205 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:39.212+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:39.290 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:39.290 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:39.290 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:39.290 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:39.290 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:39.300 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:39.318 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-05-08T08:25:39.328 INFO:teuthology.orchestra.run.smithi031.stdout:3244 read 0x23205b0 thru 0x232d45b (0xceac bytes) 2024-05-08T08:25:39.341 INFO:teuthology.orchestra.run.smithi031.stdout:3245 writesame 0x9678961 thru 0x9684ecb data_size 0x1(0xc56b bytes) 2024-05-08T08:25:39.341 INFO:teuthology.orchestra.run.smithi031.stdout:3246 read 0x4664cdc thru 0x46656d2 (0x9f7 bytes) 2024-05-08T08:25:39.342 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:39.349+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:39.343 INFO:teuthology.orchestra.run.smithi031.stdout:3248 trunc from 0x9684ecc to 0xacac47e 2024-05-08T08:25:39.348 INFO:teuthology.orchestra.run.smithi031.stdout:3249 punch from 0x98a639a to 0x98b519a, (0xee00 bytes) 2024-05-08T08:25:39.348 INFO:teuthology.orchestra.run.smithi031.stdout:3250 punch from 0x3f566e to 0x404d83, (0xf715 bytes) 2024-05-08T08:25:39.348 INFO:teuthology.orchestra.run.smithi031.stdout:3252 write 0x25e114e thru 0x25e69a5 (0x5858 bytes) 2024-05-08T08:25:39.360 INFO:teuthology.orchestra.run.smithi031.stdout:3253 write 0xe6b9764 thru 0xe6bf18e (0x5a2b bytes) 2024-05-08T08:25:39.363 INFO:teuthology.orchestra.run.smithi031.stdout:3254 write 0x69cbe78 thru 0x69ccf04 (0x108d bytes) 2024-05-08T08:25:39.365 INFO:teuthology.orchestra.run.smithi031.stdout:3256 clone 19 order 23 su 262144 sc 14 2024-05-08T08:25:39.404 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:39.411+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:39.448 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:39.448 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.843438579000001, 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098847239999999992, 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 9 2024-05-08T08:25:39.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611387+0000", 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.181e-06 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611395+0000", 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0730000000000005e-06 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611411+0000", 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6634999999999999e-05 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611427+0000", 2024-05-08T08:25:39.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5699e-05 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611486+0000", 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8692e-05 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611491+0000", 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.989e-06 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615250+0000", 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037591579999999999 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615342+0000", 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2561999999999999e-05 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615480+0000", 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013755399999999999 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.616631+0000", 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011506999999999999 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621178+0000", 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045473620000000001 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621230+0000", 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1845999999999998e-05 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621269+0000", 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9273000000000002e-05 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.454 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1826 3.3 3:c3deaffe:::rbd_data.119116cc7dd0.00000000000001e1:head [set-alloc-hint object_size 262144 write_size 262144,write 212992~8192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.203399706999999, 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0079090730000000008, 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1826 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251423+0000", 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251425+0000", 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.215e-06 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251433+0000", 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9060000000000008e-06 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251444+0000", 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0813e-05 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251449+0000", 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251489+0000", 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9774999999999998e-05 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251542+0000", 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3162999999999999e-05 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.251630+0000", 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7309e-05 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.252128+0000", 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00049835099999999996 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259274+0000", 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071460289999999999 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259308+0000", 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3679000000000001e-05 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.259332+0000", 2024-05-08T08:25:39.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4752999999999998e-05 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2374 3.1a 3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 27.733507537000001, 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011630434, 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.460 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2374 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:39.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1349999999999999e-06 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721322+0000", 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.13e-06 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721340+0000", 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7988000000000001e-05 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721348+0000", 2024-05-08T08:25:39.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9789999999999993e-06 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721408+0000", 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0168999999999999e-05 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721494+0000", 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6001000000000006e-05 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721716+0000", 2024-05-08T08:25:39.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022242200000000001 2024-05-08T08:25:39.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.722577+0000", 2024-05-08T08:25:39.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00086067299999999995 2024-05-08T08:25:39.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732885+0000", 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010308286 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732916+0000", 2024-05-08T08:25:39.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0437999999999999e-05 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732946+0000", 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0213e-05 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7094 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1565839~19667825] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.249611303000002, 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008628854, 2024-05-08T08:25:39.466 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7094 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205213+0000", 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.54e-07 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206767+0000", 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001554607 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206779+0000", 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1599e-05 2024-05-08T08:25:39.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206785+0000", 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0050000000000001e-06 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206831+0000", 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6108e-05 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206916+0000", 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5575000000000007e-05 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207099+0000", 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000182237 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209859+0000", 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027605020000000002 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213776+0000", 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003917238 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213810+0000", 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4051999999999999e-05 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213841+0000", 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0176999999999999e-05 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7095 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:39.471 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.248024464, 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0071300790000000001, 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7095 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206799+0000", 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206801+0000", 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3429999999999998e-06 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206810+0000", 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8389999999999996e-06 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206814+0000", 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1119999999999998e-06 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208857+0000", 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020428310000000002 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.208905+0000", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7849999999999998e-05 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209120+0000", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000215362 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209862+0000", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074158700000000004 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213871+0000", 2024-05-08T08:25:39.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040092579999999999 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213903+0000", 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1661000000000001e-05 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213929+0000", 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5795e-05 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7096 3.1a 3:5aec0c35:::rbd_object_map.119120fdbc10:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:39.476 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.24751745, 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072441390000000001, 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7096 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:39.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207306+0000", 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207309+0000", 2024-05-08T08:25:39.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6960000000000001e-06 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207315+0000", 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7700000000000004e-06 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207319+0000", 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8840000000000004e-06 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209378+0000", 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020587029999999998 2024-05-08T08:25:39.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209422+0000", 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4320000000000003e-05 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209632+0000", 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020937899999999999 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.210465+0000", 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083359500000000002 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214459+0000", 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0039939270000000004 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214514+0000", 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4338000000000001e-05 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.214550+0000", 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6130000000000001e-05 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.183445958, 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:25:39.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:25:39.486 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:7160 3.1a e28/18 ondisk, result = 0)", 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.167913675000001, 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0078836489999999995, 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:39.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286909+0000", 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286911+0000", 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4580000000000001e-06 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286918+0000", 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9750000000000001e-06 2024-05-08T08:25:39.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286935+0000", 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7337e-05 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.286946+0000", 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0611999999999999e-05 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294654+0000", 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.007707958 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294664+0000", 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0475999999999999e-05 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294793+0000", 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012883299999999999 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:13529 3.14 3:29884244:::rbd_data.119174045df.0000000000000089:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:39.490 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.166107073999999, 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016452866999999999, 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 13529 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288717+0000", 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288721+0000", 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9489999999999998e-06 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288730+0000", 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1670000000000005e-06 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288735+0000", 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288772+0000", 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7175000000000002e-05 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288852+0000", 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.291616+0000", 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027647800000000001 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305030+0000", 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013413625 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305098+0000", 2024-05-08T08:25:39.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7991999999999995e-05 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305136+0000", 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8197e-05 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305169+0000", 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2601000000000002e-05 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14728 3.14 3:2f5e214b:::rbd_data.119174045df.00000000000000c9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:39.495 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.070717655999999, 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082457929999999995, 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14728 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384106+0000", 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6300000000000004e-07 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384110+0000", 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2400000000000001e-06 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384121+0000", 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0190999999999999e-05 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384126+0000", 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384173+0000", 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299000000000002e-05 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384253+0000", 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9802000000000006e-05 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.385620+0000", 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001366843 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392233+0000", 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066128469999999998 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392250+0000", 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7105999999999999e-05 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392301+0000", 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1453999999999999e-05 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392351+0000", 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9883000000000003e-05 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14789 3.9 3:90dc9cea:::rbd_data.119174045df.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.998347906999999, 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024649007000000001, 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14789 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456476+0000", 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0279999999999999e-06 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456480+0000", 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1280000000000001e-06 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456491+0000", 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0436e-05 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456497+0000", 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.64e-06 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456545+0000", 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7432999999999998e-05 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456635+0000", 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9859999999999997e-05 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.460437+0000", 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038026510000000002 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481041+0000", 2024-05-08T08:25:39.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020603652 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481053+0000", 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2301e-05 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481088+0000", 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4669999999999998e-05 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481124+0000", 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6208000000000003e-05 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.505 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:14789 3.9 e37/18 ondisk, result = 0)", 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.981243496999999, 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074674299999999997, 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:39.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473580+0000", 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473581+0000", 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.232e-06 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473585+0000", 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3309999999999997e-06 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473599+0000", 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3859000000000001e-05 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.473608+0000", 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7509999999999997e-06 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481025+0000", 2024-05-08T08:25:39.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074177699999999997 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481037+0000", 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1134e-05 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481047+0000", 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0353000000000001e-05 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.509 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15094 3.9 3:97b56102:::rbd_data.119174045df.0000000000000061:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.642339955000001, 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019831228999999999, 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15094 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812484+0000", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9700000000000005e-07 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812489+0000", 2024-05-08T08:25:39.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4559999999999999e-06 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812502+0000", 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2493999999999999e-05 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812509+0000", 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3919999999999997e-06 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812564+0000", 2024-05-08T08:25:39.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4565999999999999e-05 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812693+0000", 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012915299999999999 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.816118+0000", 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003425329 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832226+0000", 2024-05-08T08:25:39.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016107360000000001 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832238+0000", 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2500999999999999e-05 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832279+0000", 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0737e-05 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832314+0000", 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5343999999999999e-05 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15144 3.3 3:c762a68c:::rbd_data.119174045df.0000000000000027:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.568111011999999, 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099787780000000006, 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.515 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15144 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886713+0000", 2024-05-08T08:25:39.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4200000000000005e-07 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886717+0000", 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4349999999999999e-06 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886728+0000", 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0535e-05 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886735+0000", 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2139999999999999e-06 2024-05-08T08:25:39.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886779+0000", 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4382999999999998e-05 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886827+0000", 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7981e-05 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.888464+0000", 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001636637 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896530+0000", 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0080662830000000005 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896580+0000", 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9484000000000003e-05 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896642+0000", 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896691+0000", 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8848999999999999e-05 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:21267 3.1c e40/18 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:head v 40'96, mlcod=40'94)", 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.532434+0000", 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.9223889209999996, 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015531513, 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.532434+0000", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.532434+0000", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.532437+0000", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6299999999999998e-06 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.538927+0000", 2024-05-08T08:25:39.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0064904730000000001 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.538953+0000", 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5851999999999999e-05 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.538965+0000", 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1771e-05 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.539032+0000", 2024-05-08T08:25:39.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7321000000000001e-05 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.539052+0000", 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9819000000000001e-05 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.547910+0000", 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088581749999999994 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.547966+0000", 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5472e-05 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.523 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23289 3.3 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9352060959999999, 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.028538101999999999, 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23289 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.524 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519618+0000", 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3170000000000001e-06 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522813+0000", 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031947130000000001 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522829+0000", 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6005e-05 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522837+0000", 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8990000000000001e-06 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522902+0000", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4939999999999998e-05 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523060+0000", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000158085 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523351+0000", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00029110399999999998 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533281+0000", 2024-05-08T08:25:39.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099297550000000002 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548076+0000", 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014795605999999999 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548119+0000", 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2993e-05 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548155+0000", 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5685000000000001e-05 2024-05-08T08:25:39.528 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:23287 3.13 e41/18 3:ced44c57:::rbd_data.1191ea0f1b1f.0000000000000006:head v 41'439, mlcod=40'437)", 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.525280+0000", 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9295426740000003, 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084857349999999995, 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.525280+0000", 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.525280+0000", 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.525284+0000", 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7740000000000002e-06 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527782+0000", 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002498297 2024-05-08T08:25:39.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527799+0000", 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6651e-05 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527809+0000", 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0169999999999999e-05 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527877+0000", 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.813e-05 2024-05-08T08:25:39.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527908+0000", 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0660000000000001e-05 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533725+0000", 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058165810000000004 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533766+0000", 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1471999999999999e-05 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.532 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23675 3.9 3:93098200:::rbd_data.1191ea0f1b1f.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.4502467110000001, 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.027687956, 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23675 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.533 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004578+0000", 2024-05-08T08:25:39.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.198e-06 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004583+0000", 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5740000000000003e-06 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004593+0000", 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0127e-05 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004599+0000", 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:25:39.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004650+0000", 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999997e-05 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004699+0000", 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9186e-05 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.009797+0000", 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050987319999999999 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032181+0000", 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022383157000000001 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032193+0000", 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2561999999999999e-05 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032228+0000", 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4332000000000003e-05 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032264+0000", 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6797e-05 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:23675 3.9 e41/18 ondisk, result = 0)", 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.023913+0000", 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.4309103489999995, 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082743950000000004, 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023913+0000", 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023913+0000", 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023914+0000", 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.161e-06 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023919+0000", 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023937+0000", 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8181000000000001e-05 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023946+0000", 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5070000000000006e-06 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032168+0000", 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082216300000000006 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032176+0000", 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3040000000000005e-06 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032187+0000", 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0971e-05 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25040 3.9 3:919a1bc9:::rbd_data.11911e381319.0000000000000137:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.499327+0000", 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.9554959119999999, 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016015854, 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25040 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:39.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499327+0000", 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499327+0000", 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499328+0000", 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.037e-06 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499333+0000", 2024-05-08T08:25:39.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499342+0000", 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8559999999999996e-06 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499347+0000", 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0390000000000004e-06 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499398+0000", 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0699999999999999e-05 2024-05-08T08:25:39.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499476+0000", 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8003999999999994e-05 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.502263+0000", 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027868490000000001 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515245+0000", 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.012982118000000001 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515277+0000", 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2277999999999998e-05 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515312+0000", 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5488999999999999e-05 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515343+0000", 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0582e-05 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.546 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.547 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:39.547 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:39.547 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:39.548 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:39.548 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:39.555+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:39.649 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:39.656+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:39.695 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:39.695 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:39.695 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:39.695 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:39.703 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:39.787 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:39.795+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:39.849 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:39.849 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:39.849 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:39.849 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:39.849 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:39.850 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:39.858+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:39.858 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:39.921 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:39.965 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:39.972+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21267 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:40.005 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.498855132999999, 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044009312000000002, 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21267 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513028+0000", 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.598e-06 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525553+0000", 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.012524067999999999 2024-05-08T08:25:40.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525577+0000", 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4771999999999999e-05 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525605+0000", 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7577e-05 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525700+0000", 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5251e-05 2024-05-08T08:25:40.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525885+0000", 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018491199999999999 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.530129+0000", 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042443949999999998 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556920+0000", 2024-05-08T08:25:40.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02679107 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556958+0000", 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7644000000000001e-05 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556998+0000", 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0216999999999998e-05 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.557036+0000", 2024-05-08T08:25:40.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7808000000000001e-05 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.351054153, 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.011 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:25:40.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:25:40.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23217 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 32505856~362265] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5534339539999991, 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030629341000000001, 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23217 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.017 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8099999999999998e-07 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458450+0000", 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0949999999999998e-06 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458457+0000", 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4130000000000003e-06 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458461+0000", 2024-05-08T08:25:40.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3440000000000003e-06 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.488227+0000", 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029765438000000002 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.488300+0000", 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3501e-05 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.489077+0000", 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00077716899999999997 2024-05-08T08:25:40.020 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23222 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 19922944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5521911530000008, 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030552035000000002, 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.021 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23222 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459693+0000", 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459700+0000", 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1210000000000003e-06 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459704+0000", 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4839999999999998e-06 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490142+0000", 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030438302 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490186+0000", 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3656999999999999e-05 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490243+0000", 2024-05-08T08:25:40.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6891000000000001e-05 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23223 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 24117248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.552171607, 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030732103, 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23223 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459711+0000", 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459716+0000", 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2290000000000002e-06 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459730+0000", 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3947999999999999e-05 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459735+0000", 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3839999999999996e-06 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490276+0000", 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030540421000000002 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490351+0000", 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5159000000000003e-05 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490442+0000", 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1607000000000006e-05 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23224 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 28311552~452228] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5521392540000001, 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031565682999999997, 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.029 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23224 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:40.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459746+0000", 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4490000000000002e-06 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459752+0000", 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5370000000000003e-06 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459759+0000", 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490471+0000", 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030712198999999999 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490521+0000", 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9820000000000001e-05 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491308+0000", 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00078766700000000003 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23225 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 32505856~225280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:40.033 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5521166960000006, 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032237982999999998, 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23225 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:40.034 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459766+0000", 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.72e-07 2024-05-08T08:25:40.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.037 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459769+0000", 2024-05-08T08:25:40.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1200000000000002e-06 2024-05-08T08:25:40.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459774+0000", 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3669999999999996e-06 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459779+0000", 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491344+0000", 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031565054000000002 2024-05-08T08:25:40.038 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491416+0000", 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1921000000000005e-05 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492003+0000", 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058739700000000003 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.039 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23245 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 16252928~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5501531719999999, 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030930742000000001, 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23245 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.040 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7800000000000001e-07 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461732+0000", 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:40.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461738+0000", 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7490000000000001e-06 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461744+0000", 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1429999999999997e-06 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492140+0000", 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030396128000000001 2024-05-08T08:25:40.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492215+0000", 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5493000000000004e-05 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492660+0000", 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00044440799999999998 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23246 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:40.043 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5501317629999996, 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031032365999999999, 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23246 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461751+0000", 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.75e-07 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461753+0000", 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461758+0000", 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461763+0000", 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492690+0000", 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030926571999999999 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492731+0000", 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1109999999999998e-05 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492783+0000", 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1703999999999998e-05 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23247 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 24641536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5501126939999992, 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031211295999999999, 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.047 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23247 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:40.048 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461770+0000", 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461772+0000", 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9640000000000002e-06 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461777+0000", 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:25:40.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461782+0000", 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6829999999999999e-06 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492814+0000", 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031032257000000001 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492888+0000", 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3706000000000002e-05 2024-05-08T08:25:40.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492981+0000", 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2789999999999998e-05 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23248 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5500945129999995, 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031307201999999999, 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.051 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23248 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.052 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461788+0000", 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461790+0000", 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2189999999999998e-06 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461796+0000", 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7690000000000002e-06 2024-05-08T08:25:40.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461800+0000", 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6179999999999997e-06 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493006+0000", 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031205172 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493044+0000", 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8244000000000002e-05 2024-05-08T08:25:40.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493095+0000", 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0776000000000003e-05 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23249 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 33030144~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.550075605, 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032220515999999998, 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.055 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23249 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.056 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461807+0000", 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1699999999999999e-07 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461809+0000", 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461814+0000", 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7780000000000004e-06 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.057 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461819+0000", 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493127+0000", 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031307802000000003 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493194+0000", 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7335000000000005e-05 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.058 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494027+0000", 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083252399999999996 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23250 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5500543409999992, 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031361471000000002, 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.059 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23250 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.060 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461859+0000", 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0644e-05 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461866+0000", 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5900000000000002e-06 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461872+0000", 2024-05-08T08:25:40.061 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493110+0000", 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031238542000000001 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493144+0000", 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3828999999999997e-05 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493189+0000", 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4808999999999997e-05 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.062 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23252 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5499972050000004, 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031396697000000001, 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23252 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.063 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7999999999999995e-07 2024-05-08T08:25:40.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461888+0000", 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.869e-06 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461894+0000", 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1700000000000002e-06 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461899+0000", 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8359999999999999e-06 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493202+0000", 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031302950000000003 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493237+0000", 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5088e-05 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493281+0000", 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4203999999999999e-05 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23253 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 12058624~336650] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5499760400000007, 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031833767999999998, 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23253 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461909+0000", 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9739999999999998e-06 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461917+0000", 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461921+0000", 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6909999999999996e-06 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493295+0000", 2024-05-08T08:25:40.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031373511999999999 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493328+0000", 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3068000000000003e-05 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493740+0000", 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00041189299999999998 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23254 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 16252928~115013] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.549954112, 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032318688999999998, 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23254 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.071 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461930+0000", 2024-05-08T08:25:40.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2129999999999998e-06 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461937+0000", 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4840000000000001e-06 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461941+0000", 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5589999999999999e-06 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493755+0000", 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031813266999999999 2024-05-08T08:25:40.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493796+0000", 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0747999999999997e-05 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494247+0000", 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00045104900000000002 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.074 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23255 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5499133520000008, 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032478619, 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23255 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:40.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461972+0000", 2024-05-08T08:25:40.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.649e-06 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461979+0000", 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0119999999999999e-06 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461984+0000", 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2580000000000003e-06 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494282+0000", 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032298329000000001 2024-05-08T08:25:40.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494356+0000", 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3807999999999995e-05 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494447+0000", 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1153e-05 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23256 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 24641536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.078 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5498911159999995, 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032599534999999999, 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23256 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.58e-07 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461994+0000", 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3819999999999998e-06 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462001+0000", 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4580000000000004e-06 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462006+0000", 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9599999999999999e-06 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494486+0000", 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032479959000000003 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.081 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494530+0000", 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3825999999999997e-05 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494590+0000", 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0591999999999999e-05 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23257 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5498080430000005, 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032656203000000002, 2024-05-08T08:25:40.082 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23257 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462077+0000", 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3199999999999998e-06 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462083+0000", 2024-05-08T08:25:40.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9739999999999999e-06 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462088+0000", 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3619999999999997e-06 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494622+0000", 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032533524000000001 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494669+0000", 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7939999999999998e-05 2024-05-08T08:25:40.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494730+0000", 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0639e-05 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23258 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 33030144~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.5496836819999995, 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.033188701000000001, 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:40.086 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23258 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462199+0000", 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.92e-07 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462202+0000", 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462211+0000", 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4190000000000002e-06 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462217+0000", 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5570000000000001e-06 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494758+0000", 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032540628000000002 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494800+0000", 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1767999999999997e-05 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:40.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.495387+0000", 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058742199999999999 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:40.090 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:40.091 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:40.091 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:40.092 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:40.092 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:40.093 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:40.099+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:40.093 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:40.093 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:40.098 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:25:40.109 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:40.109 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:40.129 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:40.149 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:40.149 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:40.167 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:40.167 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:40.170 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:40.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:40.186 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone18 from 0xe6bf18f (overlap 0x621b7f0) to 0x4e7c659 2024-05-08T08:25:40.191 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:40.198+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:40.242 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:40.243 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:40.252 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:40.292 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:40.298+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:40.299 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-05-08T08:25:40.300 INFO:teuthology.orchestra.run.smithi031.stdout:3257 punch from 0x6269d54 to 0x6274cc7, (0xaf73 bytes) 2024-05-08T08:25:40.304 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.304 INFO:teuthology.orchestra.run.smithi031.stdout:3259 read 0x2dc5c96 thru 0x2dccf0f (0x727a bytes) 2024-05-08T08:25:40.304 INFO:teuthology.orchestra.run.smithi031.stdout:3260 read 0x5895cbd thru 0x589db1a (0x7e5e bytes) 2024-05-08T08:25:40.304 INFO:teuthology.orchestra.run.smithi031.stdout:3262 read 0x7c59853 thru 0x7c68da2 (0xf550 bytes) 2024-05-08T08:25:40.304 INFO:teuthology.orchestra.run.smithi031.stdout:3265 writesame 0x639a85a thru 0x639e657 data_size 0x45(0x3dfe bytes) 2024-05-08T08:25:40.330 INFO:teuthology.orchestra.run.smithi031.stdout:3266 read 0x94fff74 thru 0x9509b0c (0x9b99 bytes) 2024-05-08T08:25:40.330 INFO:teuthology.orchestra.run.smithi031.stdout:3268 write 0x755ebd7 thru 0x756c7ee (0xdc18 bytes) 2024-05-08T08:25:40.342 INFO:teuthology.orchestra.run.smithi031.stdout:3270 trunc from 0xe6bf18f to 0x6dfd787 2024-05-08T08:25:40.371 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:40.371 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:40.384 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:40.393 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:40.399+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:40.431 INFO:teuthology.orchestra.run.smithi031.stdout:3271 read 0xdb6404 thru 0xdbba39 (0x5636 bytes) 2024-05-08T08:25:40.432 INFO:teuthology.orchestra.run.smithi031.stdout:3272 read 0x2f9f3e2 thru 0x2fa0d62 (0x1981 bytes) 2024-05-08T08:25:40.432 INFO:teuthology.orchestra.run.smithi031.stdout:3273 punch from 0x644acaa to 0x6452f72, (0x82c8 bytes) 2024-05-08T08:25:40.433 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.433 INFO:teuthology.orchestra.run.smithi031.stdout:3275 read 0x464dc9e thru 0x465d4ab (0xf80e bytes) 2024-05-08T08:25:40.440 INFO:teuthology.orchestra.run.smithi031.stdout:3277 trunc from 0x6dfd787 to 0xd0a374c 2024-05-08T08:25:40.446 INFO:teuthology.orchestra.run.smithi031.stdout:3278 read 0x59513a9 thru 0x596067e (0xf2d6 bytes) 2024-05-08T08:25:40.446 INFO:teuthology.orchestra.run.smithi031.stdout:3279 writesame 0xce5111f thru 0xce5d628 data_size 0x15(0xc50a bytes) 2024-05-08T08:25:40.450 INFO:teuthology.orchestra.run.smithi031.stdout:3280 write 0xc9f083b thru 0xc9f6a2d (0x61f3 bytes) 2024-05-08T08:25:40.454 INFO:teuthology.orchestra.run.smithi031.stdout:3281 punch from 0xb2950a3 to 0xb29cae4, (0x7a41 bytes) 2024-05-08T08:25:40.454 INFO:teuthology.orchestra.run.smithi031.stdout:3282 writesame 0x4bcb44 thru 0x4cb882 data_size 0x5(0xed3f bytes) 2024-05-08T08:25:40.477 INFO:teuthology.orchestra.run.smithi031.stdout:3283 trunc from 0xd0a374c to 0xda26330 2024-05-08T08:25:40.484 INFO:teuthology.orchestra.run.smithi031.stdout:3284 trunc from 0xda26330 to 0xcd27503 2024-05-08T08:25:40.493 INFO:teuthology.orchestra.run.smithi031.stdout:3286 trunc from 0xcd27503 to 0xa71b441 2024-05-08T08:25:40.493 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:40.501+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:40.501 INFO:teuthology.orchestra.run.smithi031.stdout:3287 read 0xa3825b3 thru 0xa384e97 (0x28e5 bytes) 2024-05-08T08:25:40.502 INFO:teuthology.orchestra.run.smithi031.stdout:3289 trunc from 0xa71b441 to 0x8b5a1b4 2024-05-08T08:25:40.509 INFO:teuthology.orchestra.run.smithi031.stdout:3290 read 0x7045810 thru 0x7046888 (0x1079 bytes) 2024-05-08T08:25:40.509 INFO:teuthology.orchestra.run.smithi031.stdout:3291 punch from 0x1ae62ee to 0x1af01f9, (0x9f0b bytes) 2024-05-08T08:25:40.512 INFO:teuthology.orchestra.run.smithi031.stdout:3292 trunc from 0x8b5a1b4 to 0x53536ce 2024-05-08T08:25:40.530 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:40.530 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:40.550 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:40.555 INFO:teuthology.orchestra.run.smithi031.stdout:3293 read 0x3061db0 thru 0x30663d7 (0x4628 bytes) 2024-05-08T08:25:40.556 INFO:teuthology.orchestra.run.smithi031.stdout:3294 trunc from 0x53536ce to 0x511cc43 2024-05-08T08:25:40.564 INFO:teuthology.orchestra.run.smithi031.stdout:3295 read 0x48411b3 thru 0x4845bfb (0x4a49 bytes) 2024-05-08T08:25:40.576 INFO:teuthology.orchestra.run.smithi031.stdout:3296 write 0xa4e5922 thru 0xa4f40d1 (0xe7b0 bytes) 2024-05-08T08:25:40.579 INFO:teuthology.orchestra.run.smithi031.stdout:3298 writesame 0x5602cca thru 0x561028d data_size 0x2(0xd5c4 bytes) 2024-05-08T08:25:40.592 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.592 INFO:teuthology.orchestra.run.smithi031.stdout:3301 read 0x889f703 thru 0x88a645f (0x6d5d bytes) 2024-05-08T08:25:40.592 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.592 INFO:teuthology.orchestra.run.smithi031.stdout:3304 writesame 0x338567 thru 0x3477b7 data_size 0x1(0xf251 bytes) 2024-05-08T08:25:40.617 INFO:teuthology.orchestra.run.smithi031.stdout:3306 writesame 0xdda61ff thru 0xddb22ba data_size 0x5(0xc0bc bytes) 2024-05-08T08:25:40.617 INFO:teuthology.orchestra.run.smithi031.stdout:3307 punch from 0xc8f61d5 to 0xc8fb06b, (0x4e96 bytes) 2024-05-08T08:25:40.617 INFO:teuthology.orchestra.run.smithi031.stdout:3309 punch from 0x1cd3497 to 0x1cd75b3, (0x411c bytes) 2024-05-08T08:25:40.617 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.617 INFO:teuthology.orchestra.run.smithi031.stdout:3312 write 0xb921074 thru 0xb92cdd2 (0xbd5f bytes) 2024-05-08T08:25:40.620 INFO:teuthology.orchestra.run.smithi031.stdout:3313 write 0xa92c564 thru 0xa93694e (0xa3eb bytes) 2024-05-08T08:25:40.625 INFO:teuthology.orchestra.run.smithi031.stdout:3314 writesame 0xb3c9d2d thru 0xb3cb214 data_size 0xc(0x14e8 bytes) 2024-05-08T08:25:40.627 INFO:teuthology.orchestra.run.smithi031.stdout:3315 write 0x472723e thru 0x4731bfc (0xa9bf bytes) 2024-05-08T08:25:40.629 INFO:teuthology.orchestra.run.smithi031.stdout:3317 write 0x65d5274 thru 0x65e1d60 (0xcaed bytes) 2024-05-08T08:25:40.630 INFO:teuthology.orchestra.run.smithi031.stdout:3318 read 0x9f1ba27 thru 0x9f1e492 (0x2a6c bytes) 2024-05-08T08:25:40.630 INFO:teuthology.orchestra.run.smithi031.stdout:3319 read 0x61ca332 thru 0x61d02eb (0x5fba bytes) 2024-05-08T08:25:40.631 INFO:teuthology.orchestra.run.smithi031.stdout:3320 writesame 0x2f3806d thru 0x2f3b85e data_size 0x21(0x37f2 bytes) 2024-05-08T08:25:40.632 INFO:teuthology.orchestra.run.smithi031.stdout:3324 punch from 0xb4f997a to 0xb50430c, (0xa992 bytes) 2024-05-08T08:25:40.632 INFO:teuthology.orchestra.run.smithi031.stdout:3325 write 0x574cf8c thru 0x575659a (0x960f bytes) 2024-05-08T08:25:40.634 INFO:teuthology.orchestra.run.smithi031.stdout:3326 read 0xb1be4ed thru 0xb1c7c5f (0x9773 bytes) 2024-05-08T08:25:40.635 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:40.643+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:40.640 INFO:teuthology.orchestra.run.smithi031.stdout:3328 trunc from 0xddb22bb to 0x8666322 2024-05-08T08:25:40.647 INFO:teuthology.orchestra.run.smithi031.stdout:3330 writesame 0x3a6f154 thru 0x3a79417 data_size 0x16(0xa2c4 bytes) 2024-05-08T08:25:40.649 INFO:teuthology.orchestra.run.smithi031.stdout:3331 writesame 0x3810a82 thru 0x38165bc data_size 0x2d(0x5b3b bytes) 2024-05-08T08:25:40.650 INFO:teuthology.orchestra.run.smithi031.stdout:3332 writesame 0x62cfdd1 thru 0x62ddd11 data_size 0x1(0xdf41 bytes) 2024-05-08T08:25:40.676 INFO:teuthology.orchestra.run.smithi031.stdout:3333 punch from 0x7cdfd47 to 0x7cecd9d, (0xd056 bytes) 2024-05-08T08:25:40.689 INFO:teuthology.orchestra.run.smithi031.stdout:3334 write 0xdd86beb thru 0xdd921ca (0xb5e0 bytes) 2024-05-08T08:25:40.690 INFO:teuthology.orchestra.run.smithi031.stdout:3335 writesame 0x9f45bf2 thru 0x9f51521 data_size 0x8(0xb930 bytes) 2024-05-08T08:25:40.694 INFO:teuthology.orchestra.run.smithi031.stdout:3336 punch from 0x7730c2 to 0x779d89, (0x6cc7 bytes) 2024-05-08T08:25:40.694 INFO:teuthology.orchestra.run.smithi031.stdout:3337 write 0x6eff47f thru 0x6f0d077 (0xdbf9 bytes) 2024-05-08T08:25:40.695 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:40.702+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:40.700 INFO:teuthology.orchestra.run.smithi031.stdout:3338 read 0xb7e0853 thru 0xb7f0350 (0xfafe bytes) 2024-05-08T08:25:40.701 INFO:teuthology.orchestra.run.smithi031.stdout:3339 writesame 0xd2a58f5 thru 0xd2a7d93 data_size 0x5(0x249f bytes) 2024-05-08T08:25:40.705 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:40.705 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:40.709 INFO:teuthology.orchestra.run.smithi031.stdout:3340 trunc from 0xdd921cb to 0x9168993 2024-05-08T08:25:40.715 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:40.717 INFO:teuthology.orchestra.run.smithi031.stdout:3341 read 0x768fbff thru 0x76938c2 (0x3cc4 bytes) 2024-05-08T08:25:40.719 INFO:teuthology.orchestra.run.smithi031.stdout:3342 punch from 0x19f00ac to 0x19f8c42, (0x8b96 bytes) 2024-05-08T08:25:40.719 INFO:teuthology.orchestra.run.smithi031.stdout:3343 write 0x2fe60a0 thru 0x2fef6fa (0x965b bytes) 2024-05-08T08:25:40.721 INFO:teuthology.orchestra.run.smithi031.stdout:3344 punch from 0x7a757e3 to 0x7a82361, (0xcb7e bytes) 2024-05-08T08:25:40.721 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.721 INFO:teuthology.orchestra.run.smithi031.stdout:3346 writesame 0x529cc5a thru 0x52a69ac data_size 0x4b(0x9d53 bytes) 2024-05-08T08:25:40.733 INFO:teuthology.orchestra.run.smithi031.stdout:3348 write 0xcb46ecb thru 0xcb4c5cd (0x5703 bytes) 2024-05-08T08:25:40.735 INFO:teuthology.orchestra.run.smithi031.stdout:3350 writesame 0x1527802 thru 0x1537450 data_size 0x1(0xfc4f bytes) 2024-05-08T08:25:40.764 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.764 INFO:teuthology.orchestra.run.smithi031.stdout:3354 write 0x4b16b49 thru 0x4b1d8be (0x6d76 bytes) 2024-05-08T08:25:40.766 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.766 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.766 INFO:teuthology.orchestra.run.smithi031.stdout:3358 writesame 0x7e403b4 thru 0x7e4db3d data_size 0x2(0xd78a bytes) 2024-05-08T08:25:40.778 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.778 INFO:teuthology.orchestra.run.smithi031.stdout:3361 read 0x23b3096 thru 0x23c263a (0xf5a5 bytes) 2024-05-08T08:25:40.778 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.778 INFO:teuthology.orchestra.run.smithi031.stdout:3363 write 0xa3ee013 thru 0xa3fc873 (0xe861 bytes) 2024-05-08T08:25:40.784 INFO:teuthology.orchestra.run.smithi031.stdout:3364 trunc from 0xcb4c5ce to 0xb0548c8 2024-05-08T08:25:40.791 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.791 INFO:teuthology.orchestra.run.smithi031.stdout:3366 punch from 0x1775f44 to 0x17820b9, (0xc175 bytes) 2024-05-08T08:25:40.791 INFO:teuthology.orchestra.run.smithi031.stdout:3367 writesame 0x942e63 thru 0x95060b data_size 0x4d(0xd7a9 bytes) 2024-05-08T08:25:40.792 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.792 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.793 INFO:teuthology.orchestra.run.smithi031.stdout:3371 write 0x280792e thru 0x281297f (0xb052 bytes) 2024-05-08T08:25:40.799 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:40.807+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:40.802 INFO:teuthology.orchestra.run.smithi031.stdout:3372 trunc from 0xb0548c8 to 0x3f9c9dc 2024-05-08T08:25:40.817 INFO:teuthology.orchestra.run.smithi031.stdout:3373 read 0x4a1a91 thru 0x4ad846 (0xbdb6 bytes) 2024-05-08T08:25:40.818 INFO:teuthology.orchestra.run.smithi031.stdout:3374 write 0xf45b97 thru 0xf53675 (0xdadf bytes) 2024-05-08T08:25:40.820 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.827 INFO:teuthology.orchestra.run.smithi031.stdout:3376 trunc from 0x3f9c9dc to 0xb0662ae 2024-05-08T08:25:40.833 INFO:teuthology.orchestra.run.smithi031.stdout:3377 write 0x97b6ce7 thru 0x97c0198 (0x94b2 bytes) 2024-05-08T08:25:40.836 INFO:teuthology.orchestra.run.smithi031.stdout:3378 writesame 0xad553dd thru 0xad5b119 data_size 0x1(0x5d3d bytes) 2024-05-08T08:25:40.851 INFO:teuthology.orchestra.run.smithi031.stdout:3379 trunc from 0xb0662ae to 0x69da55d 2024-05-08T08:25:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.864 INFO:teuthology.orchestra.run.smithi031.stdout:3382 writesame 0x4cecd05 thru 0x4cf47a4 data_size 0x9(0x7aa0 bytes) 2024-05-08T08:25:40.866 INFO:teuthology.orchestra.run.smithi031.stdout:3383 read 0x10982ff thru 0x10a37a5 (0xb4a7 bytes) 2024-05-08T08:25:40.867 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.867 INFO:teuthology.orchestra.run.smithi031.stdout:3386 punch from 0x5e72c4b to 0x5e78019, (0x53ce bytes) 2024-05-08T08:25:40.872 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:40.872 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:40.880 INFO:teuthology.orchestra.run.smithi031.stdout:3387 write 0xb930791 thru 0xb93942f (0x8c9f bytes) 2024-05-08T08:25:40.881 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:40.884 INFO:teuthology.orchestra.run.smithi031.stdout:3389 punch from 0x11ba071 to 0x11c023f, (0x61ce bytes) 2024-05-08T08:25:40.884 INFO:teuthology.orchestra.run.smithi031.stdout:3390 punch from 0x9e5c373 to 0x9e6a202, (0xde8f bytes) 2024-05-08T08:25:40.884 INFO:teuthology.orchestra.run.smithi031.stdout:3391 read 0x92e632a thru 0x92ee391 (0x8068 bytes) 2024-05-08T08:25:40.887 INFO:teuthology.orchestra.run.smithi031.stdout:3392 trunc from 0xb939430 to 0x8801b84 2024-05-08T08:25:40.896 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:40.903+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:40.902 INFO:teuthology.orchestra.run.smithi031.stdout:3394 trunc from 0x8801b84 to 0xdd1222 2024-05-08T08:25:40.917 INFO:teuthology.orchestra.run.smithi031.stdout:3395 read 0xa6215c thru 0xa64cee (0x2b93 bytes) 2024-05-08T08:25:40.918 INFO:teuthology.orchestra.run.smithi031.stdout:3396 punch from 0xabc268 to 0xac6ee7, (0xac7f bytes) 2024-05-08T08:25:40.931 INFO:teuthology.orchestra.run.smithi031.stdout:3397 writesame 0x7065309 thru 0x7067d51 data_size 0x5(0x2a49 bytes) 2024-05-08T08:25:40.931 INFO:teuthology.orchestra.run.smithi031.stdout:3398 write 0x6e15819 thru 0x6e19c08 (0x43f0 bytes) 2024-05-08T08:25:40.932 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.946 INFO:teuthology.orchestra.run.smithi031.stdout:3400 write 0xe5e8fee thru 0xe5edf89 (0x4f9c bytes) 2024-05-08T08:25:40.949 INFO:teuthology.orchestra.run.smithi031.stdout:3401 write 0xbff40ca thru 0xc000cc2 (0xcbf9 bytes) 2024-05-08T08:25:40.953 INFO:teuthology.orchestra.run.smithi031.stdout:3402 punch from 0x6570df8 to 0x6573c63, (0x2e6b bytes) 2024-05-08T08:25:40.953 INFO:teuthology.orchestra.run.smithi031.stdout:3404 read 0x7275aa2 thru 0x7281d01 (0xc260 bytes) 2024-05-08T08:25:40.954 INFO:teuthology.orchestra.run.smithi031.stdout:3405 read 0xa0c229a thru 0xa0cee88 (0xcbef bytes) 2024-05-08T08:25:40.962 INFO:teuthology.orchestra.run.smithi031.stdout:3406 trunc from 0xe5edf8a to 0x626e340 2024-05-08T08:25:40.976 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.976 INFO:teuthology.orchestra.run.smithi031.stdout:3408 punch from 0x16ed1df to 0x16f842d, (0xb24e bytes) 2024-05-08T08:25:40.989 INFO:teuthology.orchestra.run.smithi031.stdout:3409 write 0xd09b0e0 thru 0xd0a3090 (0x7fb1 bytes) 2024-05-08T08:25:40.992 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:40.992 INFO:teuthology.orchestra.run.smithi031.stdout:3412 write 0xd70839 thru 0xd7c9bf (0xc187 bytes) 2024-05-08T08:25:40.997 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:41.004+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:41.001 INFO:teuthology.orchestra.run.smithi031.stdout:3413 writesame 0xd6422f4 thru 0xd643d2d data_size 0x9(0x1a3a bytes) 2024-05-08T08:25:41.001 INFO:teuthology.orchestra.run.smithi031.stdout:3414 punch from 0x4a8c866 to 0x4a9c514, (0xfcae bytes) 2024-05-08T08:25:41.001 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:41.001 INFO:teuthology.orchestra.run.smithi031.stdout:3417 write 0x8b60733 thru 0x8b61cc5 (0x1593 bytes) 2024-05-08T08:25:41.004 INFO:teuthology.orchestra.run.smithi031.stdout:3418 punch from 0x528c0b4 to 0x529645f, (0xa3ab bytes) 2024-05-08T08:25:41.004 INFO:teuthology.orchestra.run.smithi031.stdout:3421 flatten 2024-05-08T08:25:41.010 INFO:teuthology.orchestra.run.smithi031.stdout:3422 trunc from 0xd643d2e to 0xd89f481 2024-05-08T08:25:41.016 INFO:teuthology.orchestra.run.smithi031.stdout:3423 clone 20 order 20 su 65536 sc 5 2024-05-08T08:25:41.037 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:41.037 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:41.046 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:41.097 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:41.105+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:41.164 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:41.164 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:41.174 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:41.218 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:41.225+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:41.292 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:41.298 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:41.306+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:41.302 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:41.316 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-05-08T08:25:41.399 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:41.405+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:41.428 INFO:teuthology.orchestra.run.smithi031.stdout:3424 write 0xe3a647a thru 0xe3a76a7 (0x122e bytes) 2024-05-08T08:25:41.431 INFO:teuthology.orchestra.run.smithi031.stdout:3425 writesame 0x4e7baff thru 0x4e86931 data_size 0xf(0xae33 bytes) 2024-05-08T08:25:41.438 INFO:teuthology.orchestra.run.smithi031.stdout:3426 punch from 0x3b4c7c8 to 0x3b4e0ce, (0x1906 bytes) 2024-05-08T08:25:41.442 INFO:teuthology.orchestra.run.smithi031.stdout:3427 trunc from 0xe3a76a8 to 0xa4e777c 2024-05-08T08:25:41.443 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:41.443 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:41.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:41.464 INFO:teuthology.orchestra.run.smithi031.stdout:3428 write 0xb0c7534 thru 0xb0d467b (0xd148 bytes) 2024-05-08T08:25:41.469 INFO:teuthology.orchestra.run.smithi031.stdout:3429 punch from 0xa8c0bf6 to 0xa8c2447, (0x1851 bytes) 2024-05-08T08:25:41.469 INFO:teuthology.orchestra.run.smithi031.stdout:3430 punch from 0x63c9200 to 0x63c9efa, (0xcfa bytes) 2024-05-08T08:25:41.469 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:41.469 INFO:teuthology.orchestra.run.smithi031.stdout:3433 read 0xa6988d6 thru 0xa6a649c (0xdbc7 bytes) 2024-05-08T08:25:41.470 INFO:teuthology.orchestra.run.smithi031.stdout:3434 punch from 0x2632e2e to 0x2633d55, (0xf27 bytes) 2024-05-08T08:25:41.470 INFO:teuthology.orchestra.run.smithi031.stdout:3435 punch from 0x7a1ba15 to 0x7a286ec, (0xccd7 bytes) 2024-05-08T08:25:41.470 INFO:teuthology.orchestra.run.smithi031.stdout:3436 read 0x941b7ec thru 0x9429111 (0xd926 bytes) 2024-05-08T08:25:41.470 INFO:teuthology.orchestra.run.smithi031.stdout:3437 read 0x6f961d6 thru 0x6f9bebd (0x5ce8 bytes) 2024-05-08T08:25:41.478 INFO:teuthology.orchestra.run.smithi031.stdout:3438 trunc from 0xb0d467c to 0x3e8274d 2024-05-08T08:25:41.499 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:41.506+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:41.502 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:41.511 INFO:teuthology.orchestra.run.smithi031.stdout:3441 writesame 0x6a2156a thru 0x6a2a3d7 data_size 0x3b(0x8e6e bytes) 2024-05-08T08:25:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:3442 punch from 0x50208c2 to 0x502ef7c, (0xe6ba bytes) 2024-05-08T08:25:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:3444 punch from 0x64d9c8c to 0x64e3c63, (0x9fd7 bytes) 2024-05-08T08:25:41.512 INFO:teuthology.orchestra.run.smithi031.stdout:3446 write 0x533b948 thru 0x534a64a (0xed03 bytes) 2024-05-08T08:25:41.523 INFO:teuthology.orchestra.run.smithi031.stdout:3448 writesame 0x7418716 thru 0x7426ee4 data_size 0x83(0xe7cf bytes) 2024-05-08T08:25:41.523 INFO:teuthology.orchestra.run.smithi031.stdout:3449 read 0x3595335 thru 0x35961be (0xe8a bytes) 2024-05-08T08:25:41.523 INFO:teuthology.orchestra.run.smithi031.stdout:3450 clone 21 order 19 su 32768 sc 12 2024-05-08T08:25:41.592 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:41.592 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:41.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:41.600 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:41.606+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:41.700 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:41.707+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:41.750 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:41.750 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:41.759 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:41.801 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:41.809+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:41.902 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:41.909+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:41.908 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:41.918 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:42.003 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:42.010+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:42.028 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:42.028 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:42.036 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:42.103 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:42.110+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:42.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:25:42.152 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:42.152 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:42.162 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:42.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:42.172 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone20 intact 2024-05-08T08:25:42.216 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:42.223+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:42.252 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:42.258+0000 7f0060411640 -1 osd.1 30 log_to_monitors true 2024-05-08T08:25:42.288 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-05-08T08:25:42.302 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:42.303 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:42.305 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:42.311+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:42.319 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:42.329 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:42.329 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:42.330 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-05-08T08:25:42.354 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:42.354 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:42.386 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:42.386 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:42.405 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:42.406 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:42.406 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:42.412+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:42.424 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:42.424 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:42.431 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:42.431 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:42.431 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:42.431 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:42.440 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:42.442 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:42.442 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:42.460 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:42.460 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:42.463 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:25:42.463 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:25:42.506 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:42.513+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:42.542 INFO:teuthology.orchestra.run.smithi031.stdout:3453 read 0x4c10ab6 thru 0x4c1c121 (0xb66c bytes) 2024-05-08T08:25:42.542 INFO:teuthology.orchestra.run.smithi031.stdout:3460 write 0x1dc71df thru 0x1dcfbec (0x8a0e bytes) 2024-05-08T08:25:42.545 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:42.545 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:42.545 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:42.546 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:42.546 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:42.556 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:42.564 INFO:teuthology.orchestra.run.smithi031.stdout:3462 writesame 0xa2b0d44 thru 0xa2b774b data_size 0x3(0x6a08 bytes) 2024-05-08T08:25:42.565 INFO:teuthology.orchestra.run.smithi031.stdout:3463 trunc from 0xa2b774c to 0xb7e626e 2024-05-08T08:25:42.571 INFO:teuthology.orchestra.run.smithi031.stdout:3465 read 0x12bbc69 thru 0x12cade8 (0xf180 bytes) 2024-05-08T08:25:42.572 INFO:teuthology.orchestra.run.smithi031.stdout:3466 write 0x282bec2 thru 0x283917f (0xd2be bytes) 2024-05-08T08:25:42.587 INFO:teuthology.orchestra.run.smithi031.stdout:3467 write 0xd22f6e8 thru 0xd23752f (0x7e48 bytes) 2024-05-08T08:25:42.592 INFO:teuthology.orchestra.run.smithi031.stdout:3469 punch from 0x6c3cd63 to 0x6c4c899, (0xfb36 bytes) 2024-05-08T08:25:42.592 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.592 INFO:teuthology.orchestra.run.smithi031.stdout:3473 write 0x18931f4 thru 0x18a022b (0xd038 bytes) 2024-05-08T08:25:42.598 INFO:teuthology.orchestra.run.smithi031.stdout:3474 read 0x4b77a48 thru 0x4b7e6c7 (0x6c80 bytes) 2024-05-08T08:25:42.599 INFO:teuthology.orchestra.run.smithi031.stdout:3475 read 0x9ca070e thru 0x9ca97b7 (0x90aa bytes) 2024-05-08T08:25:42.599 INFO:teuthology.orchestra.run.smithi031.stdout:3476 write 0x4c09d4d thru 0x4c0e989 (0x4c3d bytes) 2024-05-08T08:25:42.601 INFO:teuthology.orchestra.run.smithi031.stdout:3477 read 0x5e94ffd thru 0x5ea06ae (0xb6b2 bytes) 2024-05-08T08:25:42.602 INFO:teuthology.orchestra.run.smithi031.stdout:3478 read 0xba1b649 thru 0xba24e5e (0x9816 bytes) 2024-05-08T08:25:42.602 INFO:teuthology.orchestra.run.smithi031.stdout:3479 writesame 0x72284be thru 0x723808b data_size 0x2(0xfbce bytes) 2024-05-08T08:25:42.608 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:42.615+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:42.613 INFO:teuthology.orchestra.run.smithi031.stdout:3480 writesame 0x169440b thru 0x1698246 data_size 0xe(0x3e3c bytes) 2024-05-08T08:25:42.619 INFO:teuthology.orchestra.run.smithi031.stdout:3481 write 0x353dfe6 thru 0x3548a72 (0xaa8d bytes) 2024-05-08T08:25:42.625 INFO:teuthology.orchestra.run.smithi031.stdout:3483 writesame 0x971c538 thru 0x9723899 data_size 0x1b(0x7362 bytes) 2024-05-08T08:25:42.629 INFO:teuthology.orchestra.run.smithi031.stdout:3485 read 0x2ca31c6 thru 0x2cae473 (0xb2ae bytes) 2024-05-08T08:25:42.630 INFO:teuthology.orchestra.run.smithi031.stdout:3486 writesame 0xb6f6efd thru 0xb6fffa8 data_size 0x4(0x90ac bytes) 2024-05-08T08:25:42.636 INFO:teuthology.orchestra.run.smithi031.stdout:3487 trunc from 0xd237530 to 0xd9c8af4 2024-05-08T08:25:42.642 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.642 INFO:teuthology.orchestra.run.smithi031.stdout:3489 punch from 0x9a24c18 to 0x9a2cc44, (0x802c bytes) 2024-05-08T08:25:42.649 INFO:teuthology.orchestra.run.smithi031.stdout:3491 trunc from 0xd9c8af4 to 0x6eb3e20 2024-05-08T08:25:42.686 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:42.686 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:9 3.1a 3:5cea7035:::rbd_info:head [write 0~19] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e21)", 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.081434440999999, 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098847239999999992, 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 9 2024-05-08T08:25:42.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611384+0000", 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611387+0000", 2024-05-08T08:25:42.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.181e-06 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611395+0000", 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0730000000000005e-06 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611411+0000", 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6634999999999999e-05 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611427+0000", 2024-05-08T08:25:42.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5699e-05 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611486+0000", 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8692e-05 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op must wait for map", 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.611491+0000", 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.989e-06 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615250+0000", 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037591579999999999 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615342+0000", 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2561999999999999e-05 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.615480+0000", 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013755399999999999 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.616631+0000", 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011506999999999999 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621178+0000", 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045473620000000001 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621230+0000", 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1845999999999998e-05 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:06.621269+0000", 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9273000000000002e-05 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:2374 3.1a 3:5ce00439:::rbd_data.119116cc7dd0.0000000000000207:head [delete] snapc 4=[4] ondisk+write+known_if_redirected+supports_pool_eio e24)", 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 30.971503398999999, 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011630434, 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.693 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 2374 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721316+0000", 2024-05-08T08:25:42.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1349999999999999e-06 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721322+0000", 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.13e-06 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721340+0000", 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7988000000000001e-05 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721348+0000", 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9789999999999993e-06 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721408+0000", 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0168999999999999e-05 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721494+0000", 2024-05-08T08:25:42.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6001000000000006e-05 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.721716+0000", 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022242200000000001 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.722577+0000", 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00086067299999999995 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732885+0000", 2024-05-08T08:25:42.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010308286 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732916+0000", 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0437999999999999e-05 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:11.732946+0000", 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0213e-05 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7094 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1565839~19667825] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 26.487607165, 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008628854, 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7094 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205212+0000", 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.205213+0000", 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.54e-07 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206767+0000", 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001554607 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206779+0000", 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1599e-05 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206785+0000", 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0050000000000001e-06 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206831+0000", 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6108e-05 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.206916+0000", 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5575000000000007e-05 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.207099+0000", 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000182237 2024-05-08T08:25:42.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.209859+0000", 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027605020000000002 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213776+0000", 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003917238 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213810+0000", 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4051999999999999e-05 2024-05-08T08:25:42.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.213841+0000", 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0176999999999999e-05 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 26.421441819999998, 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.704 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.709 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:13529 3.14 3:29884244:::rbd_data.119174045df.0000000000000089:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.404102936000001, 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016452866999999999, 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 13529 2024-05-08T08:25:42.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288716+0000", 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288717+0000", 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6300000000000005e-07 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288721+0000", 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9489999999999998e-06 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288730+0000", 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1670000000000005e-06 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288735+0000", 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7790000000000003e-06 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288772+0000", 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7175000000000002e-05 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.288852+0000", 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.291616+0000", 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027647800000000001 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305030+0000", 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013413625 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305098+0000", 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7991999999999995e-05 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305136+0000", 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8197e-05 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:24.305169+0000", 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2601000000000002e-05 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14728 3.14 3:2f5e214b:::rbd_data.119174045df.00000000000000c9:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:42.715 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.308713518000001, 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082457929999999995, 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14728 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384105+0000", 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384106+0000", 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6300000000000004e-07 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384110+0000", 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2400000000000001e-06 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384121+0000", 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0190999999999999e-05 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384126+0000", 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384173+0000", 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299000000000002e-05 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.384253+0000", 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9802000000000006e-05 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.385620+0000", 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001366843 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392233+0000", 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066128469999999998 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392250+0000", 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7105999999999999e-05 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392301+0000", 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1453999999999999e-05 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.392351+0000", 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9883000000000003e-05 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14789 3.9 3:90dc9cea:::rbd_data.119174045df.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.236343769000001, 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024649007000000001, 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.721 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14789 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456476+0000", 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0279999999999999e-06 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456480+0000", 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1280000000000001e-06 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456491+0000", 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0436e-05 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456497+0000", 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.64e-06 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456545+0000", 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7432999999999998e-05 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456635+0000", 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9859999999999997e-05 2024-05-08T08:25:42.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.460437+0000", 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038026510000000002 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481041+0000", 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020603652 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481053+0000", 2024-05-08T08:25:42.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2301e-05 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481088+0000", 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4669999999999998e-05 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481124+0000", 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6208000000000003e-05 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15094 3.9 3:97b56102:::rbd_data.119174045df.0000000000000061:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.880335816999999, 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019831228999999999, 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15094 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.727 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812483+0000", 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812484+0000", 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9700000000000005e-07 2024-05-08T08:25:42.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812489+0000", 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4559999999999999e-06 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812502+0000", 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2493999999999999e-05 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812509+0000", 2024-05-08T08:25:42.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3919999999999997e-06 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812564+0000", 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4565999999999999e-05 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.812693+0000", 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012915299999999999 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:42.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.816118+0000", 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003425329 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832226+0000", 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016107360000000001 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832238+0000", 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2500999999999999e-05 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832279+0000", 2024-05-08T08:25:42.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0737e-05 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.832314+0000", 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5343999999999999e-05 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:15144 3.3 3:c762a68c:::rbd_data.119174045df.0000000000000027:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.806106874000001, 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099787780000000006, 2024-05-08T08:25:42.732 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 15144 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886712+0000", 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886713+0000", 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4200000000000005e-07 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886717+0000", 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4349999999999999e-06 2024-05-08T08:25:42.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886728+0000", 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0535e-05 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886735+0000", 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2139999999999999e-06 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886779+0000", 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4382999999999998e-05 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.886827+0000", 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7981e-05 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.888464+0000", 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001636637 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896530+0000", 2024-05-08T08:25:42.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0080662830000000005 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896580+0000", 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9484000000000003e-05 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896642+0000", 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.896691+0000", 2024-05-08T08:25:42.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8848999999999999e-05 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:21267 3.1c e40/18 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:head v 40'96, mlcod=40'94)", 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.532434+0000", 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.160384783, 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015531513, 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.738 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.532434+0000", 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.532434+0000", 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.532437+0000", 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6299999999999998e-06 2024-05-08T08:25:42.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.538927+0000", 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0064904730000000001 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.538953+0000", 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5851999999999999e-05 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.538965+0000", 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1771e-05 2024-05-08T08:25:42.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.539032+0000", 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7321000000000001e-05 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.539052+0000", 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9819000000000001e-05 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.547910+0000", 2024-05-08T08:25:42.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088581749999999994 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.547966+0000", 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5472e-05 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23289 3.3 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.173201958, 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.028538101999999999, 2024-05-08T08:25:42.742 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23289 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519618+0000", 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3170000000000001e-06 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522813+0000", 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031947130000000001 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522829+0000", 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6005e-05 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522837+0000", 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8990000000000001e-06 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522902+0000", 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4939999999999998e-05 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523060+0000", 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000158085 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523351+0000", 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00029110399999999998 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533281+0000", 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099297550000000002 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548076+0000", 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014795605999999999 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548119+0000", 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2993e-05 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548155+0000", 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5685000000000001e-05 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.747 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop(client.4497.0:23287 3.13 e41/18 3:ced44c57:::rbd_data.1191ea0f1b1f.0000000000000006:head v 41'439, mlcod=40'437)", 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.525280+0000", 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.167538536, 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084857349999999995, 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.525280+0000", 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.525280+0000", 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.525284+0000", 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7740000000000002e-06 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527782+0000", 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002498297 2024-05-08T08:25:42.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527799+0000", 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6651e-05 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527809+0000", 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0169999999999999e-05 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527877+0000", 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.813e-05 2024-05-08T08:25:42.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.527908+0000", 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0660000000000001e-05 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533725+0000", 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058165810000000004 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533766+0000", 2024-05-08T08:25:42.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1471999999999999e-05 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23675 3.9 3:93098200:::rbd_data.1191ea0f1b1f.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.688242573, 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.027687956, 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23675 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004578+0000", 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.198e-06 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004583+0000", 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5740000000000003e-06 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004593+0000", 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0127e-05 2024-05-08T08:25:42.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004599+0000", 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004650+0000", 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999997e-05 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004699+0000", 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9186e-05 2024-05-08T08:25:42.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.009797+0000", 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050987319999999999 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032181+0000", 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022383157000000001 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032193+0000", 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2561999999999999e-05 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032228+0000", 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4332000000000003e-05 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032264+0000", 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6797e-05 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_repop_reply(client.4497.0:23675 3.9 e41/18 ondisk, result = 0)", 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.023913+0000", 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.668906210999999, 2024-05-08T08:25:42.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082743950000000004, 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023913+0000", 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023913+0000", 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023914+0000", 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.161e-06 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023919+0000", 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6410000000000001e-06 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023937+0000", 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8181000000000001e-05 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.023946+0000", 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5070000000000006e-06 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032168+0000", 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082216300000000006 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032176+0000", 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3040000000000005e-06 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032187+0000", 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0971e-05 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25040 3.9 3:919a1bc9:::rbd_data.11911e381319.0000000000000137:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.499327+0000", 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.1934917739999999, 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.016015854, 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25040 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499327+0000", 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499327+0000", 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499328+0000", 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.037e-06 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499333+0000", 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9019999999999998e-06 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499342+0000", 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8559999999999996e-06 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499347+0000", 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0390000000000004e-06 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499398+0000", 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0699999999999999e-05 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.499476+0000", 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8003999999999994e-05 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.502263+0000", 2024-05-08T08:25:42.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0027868490000000001 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515245+0000", 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.012982118000000001 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515277+0000", 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2277999999999998e-05 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515312+0000", 2024-05-08T08:25:42.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5488999999999999e-05 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.515343+0000", 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0582e-05 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:30585 3.14 3:2c0584e9:::rbd_object_map.119198308290:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:42.665082+0000", 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.027736525000000001, 2024-05-08T08:25:42.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0094933819999999999, 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 30585 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665082+0000", 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665082+0000", 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665083+0000", 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6899999999999999e-07 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665085+0000", 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2989999999999999e-06 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665090+0000", 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665093+0000", 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9189999999999999e-06 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.670727+0000", 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056346299999999998 2024-05-08T08:25:42.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.670825+0000", 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7369999999999998e-05 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.671218+0000", 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00039312400000000001 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.673573+0000", 2024-05-08T08:25:42.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023553110000000001 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674482+0000", 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00090921899999999996 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674535+0000", 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3077e-05 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674576+0000", 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0302000000000001e-05 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:30586 3.14 3:2c0584e9:::rbd_object_map.119198308290:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:42.665133+0000", 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.027685983000000001, 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010378228, 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.772 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 30586 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665133+0000", 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665133+0000", 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665133+0000", 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665135+0000", 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2500000000000001e-06 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665139+0000", 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7780000000000001e-06 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665142+0000", 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8389999999999998e-06 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.672007+0000", 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0068645240000000003 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.672064+0000", 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7504999999999999e-05 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:42.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.672337+0000", 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00027302000000000001 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674587+0000", 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022496899999999999 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675419+0000", 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000832609 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675473+0000", 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3587e-05 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675511+0000", 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8167000000000002e-05 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:30587 3.14 3:2c0584e9:::rbd_object_map.119198308290:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:42.665150+0000", 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.027669076000000001, 2024-05-08T08:25:42.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010539339, 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 30587 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665150+0000", 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665150+0000", 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665150+0000", 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3200000000000001e-07 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665152+0000", 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9810000000000002e-06 2024-05-08T08:25:42.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665156+0000", 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4649999999999999e-06 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665158+0000", 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.903e-06 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.672664+0000", 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0075057830000000002 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.672744+0000", 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0212999999999999e-05 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.673105+0000", 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00036023700000000001 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675523+0000", 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024183329999999999 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675581+0000", 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8461999999999997e-05 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675635+0000", 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3525e-05 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675689+0000", 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4205e-05 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.782 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:30588 3.14 3:2c0584e9:::rbd_object_map.119198308290:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:42.665194+0000", 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.027625298999999999, 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010645964000000001, 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 30588 2024-05-08T08:25:42.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665194+0000", 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665194+0000", 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665194+0000", 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.15e-07 2024-05-08T08:25:42.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665197+0000", 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0070000000000002e-06 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665205+0000", 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9869999999999999e-06 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665211+0000", 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.519e-06 2024-05-08T08:25:42.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.673707+0000", 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084967020000000001 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.673791+0000", 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3347e-05 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674136+0000", 2024-05-08T08:25:42.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00034582599999999998 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675741+0000", 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016041899999999999 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675758+0000", 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7452000000000002e-05 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675800+0000", 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1557999999999998e-05 2024-05-08T08:25:42.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.675840+0000", 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9960999999999999e-05 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:30589 3.14 3:2c0584e9:::rbd_object_map.119198308290:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:42.665220+0000", 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.027599004, 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011798332, 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:42.788 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 30589 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665220+0000", 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665220+0000", 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:42.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665220+0000", 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8700000000000002e-07 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665223+0000", 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8109999999999999e-06 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665227+0000", 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0849999999999999e-06 2024-05-08T08:25:42.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.665230+0000", 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9629999999999998e-06 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674646+0000", 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0094156199999999995 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674724+0000", 2024-05-08T08:25:42.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7830000000000005e-05 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.674989+0000", 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00026552599999999998 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.676964+0000", 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001974927 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.676970+0000", 2024-05-08T08:25:42.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4749999999999998e-06 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.676996+0000", 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6061e-05 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.677018+0000", 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1747e-05 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:42.793 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:42.795 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:3493 writesame 0xd096074 thru 0xd099d10 data_size 0x1(0x3c9d bytes) 2024-05-08T08:25:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:3494 trunc from 0xd099d11 to 0x71ae25d 2024-05-08T08:25:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:3496 write 0x4b07d1c thru 0x4b171e9 (0xf4ce bytes) 2024-05-08T08:25:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:3497 punch from 0x3f9297d to 0x3f95fc9, (0x364c bytes) 2024-05-08T08:25:42.798 INFO:teuthology.orchestra.run.smithi031.stdout:3499 writesame 0x6bf92b2 thru 0x6c01a81 data_size 0x6a(0x87d0 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3500 read 0x5a50993 thru 0x5a5365c (0x2cca bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3501 write 0xa50226 thru 0xa56fb2 (0x6d8d bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3502 read 0x266a99a thru 0x266aeff (0x566 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3505 write 0x39a1b57 thru 0x39a69e3 (0x4e8d bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3506 writesame 0xeb9f57c thru 0xebace3d data_size 0x3e(0xd8c2 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3507 punch from 0x80f67e6 to 0x80fd1f4, (0x6a0e bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3510 read 0xcebc149 thru 0xcec3e46 (0x7cfe bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3513 read 0x2d14fea thru 0x2d2138a (0xc3a1 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3514 punch from 0x15981e1 to 0x15a63f3, (0xe212 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3515 write 0xb73cfba thru 0xb73df21 (0xf68 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3516 punch from 0xe7a7e75 to 0xe7ad6eb, (0x5876 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3517 write 0x5c83bad thru 0x5c89ba3 (0x5ff7 bytes) 2024-05-08T08:25:42.799 INFO:teuthology.orchestra.run.smithi031.stdout:3520 punch from 0x7ad284c to 0x7ad8062, (0x5816 bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3522 trunc from 0xebace3e to 0xbe02fdd 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3523 punch from 0x697317e to 0x6973b42, (0x9c4 bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3525 write 0x92c2d17 thru 0x92c3b24 (0xe0e bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3527 writesame 0x36b510e thru 0x36b7ee8 data_size 0x2b(0x2ddb bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3529 trunc from 0xbe02fdd to 0xecf93a1 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3530 read 0x9437646 thru 0x9446911 (0xf2cc bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3531 read 0x524197e thru 0x5242790 (0xe13 bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3532 write 0x3c66014 thru 0x3c6dc75 (0x7c62 bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3533 write 0x83adffe thru 0x83aeaa9 (0xaac bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3534 writesame 0x82e4aa thru 0x835945 data_size 0x22(0x749c bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3535 write 0x30c9e47 thru 0x30d63a5 (0xc55f bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3537 punch from 0x2a54598 to 0x2a5aa8d, (0x64f5 bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3538 read 0x53feeb7 thru 0x540761c (0x8766 bytes) 2024-05-08T08:25:42.800 INFO:teuthology.orchestra.run.smithi031.stdout:3539 punch from 0x2eedd8 to 0x2fa4eb, (0xb713 bytes) 2024-05-08T08:25:42.801 INFO:teuthology.orchestra.run.smithi031.stdout:3540 read 0x7d497a2 thru 0x7d5547e (0xbcdd bytes) 2024-05-08T08:25:42.801 INFO:teuthology.orchestra.run.smithi031.stdout:3541 punch from 0x416ca2e to 0x4177c80, (0xb252 bytes) 2024-05-08T08:25:42.801 INFO:teuthology.orchestra.run.smithi031.stdout:3542 read 0xda12d25 thru 0xda196b0 (0x698c bytes) 2024-05-08T08:25:42.801 INFO:teuthology.orchestra.run.smithi031.stdout:3545 write 0x5521847 thru 0x552fcdb (0xe495 bytes) 2024-05-08T08:25:42.801 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:42.801+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:3546 write 0xc9879d8 thru 0xc98b9b4 (0x3fdd bytes) 2024-05-08T08:25:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:3547 punch from 0x3c6f33b to 0x3c7b887, (0xc54c bytes) 2024-05-08T08:25:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.802 INFO:teuthology.orchestra.run.smithi031.stdout:3549 write 0x60b956e thru 0x60c1a5c (0x84ef bytes) 2024-05-08T08:25:42.806 INFO:teuthology.orchestra.run.smithi031.stdout:3550 read 0xdc65e44 thru 0xdc6a571 (0x472e bytes) 2024-05-08T08:25:42.806 INFO:teuthology.orchestra.run.smithi031.stdout:3551 read 0xa1d49be thru 0xa1da667 (0x5caa bytes) 2024-05-08T08:25:42.806 INFO:teuthology.orchestra.run.smithi031.stdout:3552 write 0xc5954a9 thru 0xc5a4946 (0xf49e bytes) 2024-05-08T08:25:42.812 INFO:teuthology.orchestra.run.smithi031.stdout:3553 write 0x1a5f165 thru 0x1a6ceb6 (0xdd52 bytes) 2024-05-08T08:25:42.818 INFO:teuthology.orchestra.run.smithi031.stdout:3555 read 0x8d3439e thru 0x8d411ee (0xce51 bytes) 2024-05-08T08:25:42.818 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.818 INFO:teuthology.orchestra.run.smithi031.stdout:3558 read 0xd8ab6c3 thru 0xd8b06cd (0x500b bytes) 2024-05-08T08:25:42.818 INFO:teuthology.orchestra.run.smithi031.stdout:3559 writesame 0xa10c3a6 thru 0xa11c39f data_size 0x5(0xfffa bytes) 2024-05-08T08:25:42.826 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.826 INFO:teuthology.orchestra.run.smithi031.stdout:3561 write 0x1ed63a thru 0x1f45ef (0x6fb6 bytes) 2024-05-08T08:25:42.836 INFO:teuthology.orchestra.run.smithi031.stdout:3562 punch from 0xb8af45e to 0xb8b5ada, (0x667c bytes) 2024-05-08T08:25:42.836 INFO:teuthology.orchestra.run.smithi031.stdout:3564 punch from 0x1675258 to 0x167d5d4, (0x837c bytes) 2024-05-08T08:25:42.836 INFO:teuthology.orchestra.run.smithi031.stdout:3565 writesame 0x393b04d thru 0x39472ee data_size 0x7(0xc2a2 bytes) 2024-05-08T08:25:42.841 INFO:teuthology.orchestra.run.smithi031.stdout:3567 read 0x30ebfd thru 0x318f58 (0xa35c bytes) 2024-05-08T08:25:42.842 INFO:teuthology.orchestra.run.smithi031.stdout:3568 read 0x2afe25a thru 0x2b07e47 (0x9bee bytes) 2024-05-08T08:25:42.842 INFO:teuthology.orchestra.run.smithi031.stdout:3569 read 0x7b9799f thru 0x7ba07b9 (0x8e1b bytes) 2024-05-08T08:25:42.842 INFO:teuthology.orchestra.run.smithi031.stdout:3570 read 0x8049b08 thru 0x804d180 (0x3679 bytes) 2024-05-08T08:25:42.843 INFO:teuthology.orchestra.run.smithi031.stdout:3571 read 0x271074d thru 0x2719408 (0x8cbc bytes) 2024-05-08T08:25:42.843 INFO:teuthology.orchestra.run.smithi031.stdout:3573 write 0x1f7fb50 thru 0x1f8e31e (0xe7cf bytes) 2024-05-08T08:25:42.857 INFO:teuthology.orchestra.run.smithi031.stdout:3574 write 0xed31c18 thru 0xed3355d (0x1946 bytes) 2024-05-08T08:25:42.860 INFO:teuthology.orchestra.run.smithi031.stdout:3575 punch from 0xdc117c6 to 0xdc13506, (0x1d40 bytes) 2024-05-08T08:25:42.860 INFO:teuthology.orchestra.run.smithi031.stdout:3576 punch from 0x9cb162c to 0x9cbdfc8, (0xc99c bytes) 2024-05-08T08:25:42.860 INFO:teuthology.orchestra.run.smithi031.stdout:3577 write 0x78d8d5e thru 0x78e6e4d (0xe0f0 bytes) 2024-05-08T08:25:42.863 INFO:teuthology.orchestra.run.smithi031.stdout:3579 punch from 0x15fe52d to 0x1608727, (0xa1fa bytes) 2024-05-08T08:25:42.864 INFO:teuthology.orchestra.run.smithi031.stdout:3580 write 0x5973876 thru 0x59740b0 (0x83b bytes) 2024-05-08T08:25:42.867 INFO:teuthology.orchestra.run.smithi031.stdout:3581 read 0x140033 thru 0x143014 (0x2fe2 bytes) 2024-05-08T08:25:42.867 INFO:teuthology.orchestra.run.smithi031.stdout:3582 write 0xec95dca thru 0xec96455 (0x68c bytes) 2024-05-08T08:25:42.870 INFO:teuthology.orchestra.run.smithi031.stdout:3583 punch from 0x336744 to 0x33a4d2, (0x3d8e bytes) 2024-05-08T08:25:42.870 INFO:teuthology.orchestra.run.smithi031.stdout:3585 punch from 0xe591eb8 to 0xe5a182a, (0xf972 bytes) 2024-05-08T08:25:42.870 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.870 INFO:teuthology.orchestra.run.smithi031.stdout:3590 write 0xab8ae04 thru 0xab92412 (0x760f bytes) 2024-05-08T08:25:42.873 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.873 INFO:teuthology.orchestra.run.smithi031.stdout:3595 punch from 0x2598597 to 0x25a604d, (0xdab6 bytes) 2024-05-08T08:25:42.873 INFO:teuthology.orchestra.run.smithi031.stdout:3596 read 0x875840e thru 0x8761020 (0x8c13 bytes) 2024-05-08T08:25:42.883 INFO:teuthology.orchestra.run.smithi031.stdout:3597 trunc from 0xed3355e to 0x5bd1307 2024-05-08T08:25:42.895 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:42.903+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:42.911 INFO:teuthology.orchestra.run.smithi031.stdout:3598 read 0x3119ca0 thru 0x31237db (0x9b3c bytes) 2024-05-08T08:25:42.912 INFO:teuthology.orchestra.run.smithi031.stdout:3599 punch from 0x5a5a113 to 0x5a5ab03, (0x9f0 bytes) 2024-05-08T08:25:42.912 INFO:teuthology.orchestra.run.smithi031.stdout:3600 punch from 0x277fad8 to 0x278054c, (0xa74 bytes) 2024-05-08T08:25:42.912 INFO:teuthology.orchestra.run.smithi031.stdout:3602 writesame 0x3dc951 thru 0x3e8c0e data_size 0x15(0xc2be bytes) 2024-05-08T08:25:42.935 INFO:teuthology.orchestra.run.smithi031.stdout:3604 writesame 0xc70602a thru 0xc712427 data_size 0x1(0xc3fe bytes) 2024-05-08T08:25:42.935 INFO:teuthology.orchestra.run.smithi031.stdout:3605 writesame 0x4a2a503 thru 0x4a2f344 data_size 0x36(0x4e42 bytes) 2024-05-08T08:25:42.936 INFO:teuthology.orchestra.run.smithi031.stdout:3608 read 0x889279 thru 0x8967d1 (0xd559 bytes) 2024-05-08T08:25:42.937 INFO:teuthology.orchestra.run.smithi031.stdout:3610 writesame 0x647ecba thru 0x648896d data_size 0x6(0x9cb4 bytes) 2024-05-08T08:25:42.944 INFO:teuthology.orchestra.run.smithi031.stdout:3611 read 0x50ead2c thru 0x50f3840 (0x8b15 bytes) 2024-05-08T08:25:42.944 INFO:teuthology.orchestra.run.smithi031.stdout:3612 write 0x839c1bd thru 0x83a4da6 (0x8bea bytes) 2024-05-08T08:25:42.950 INFO:teuthology.orchestra.run.smithi031.stdout:3613 write 0x928494e thru 0x9288ae3 (0x4196 bytes) 2024-05-08T08:25:42.956 INFO:teuthology.orchestra.run.smithi031.stdout:3614 read 0x9e49f32 thru 0x9e57a6d (0xdb3c bytes) 2024-05-08T08:25:42.956 INFO:teuthology.orchestra.run.smithi031.stdout:3615 read 0x72d6ab5 thru 0x72d7d57 (0x12a3 bytes) 2024-05-08T08:25:42.956 INFO:teuthology.orchestra.run.smithi031.stdout:3618 write 0x962b9ab thru 0x9636429 (0xaa7f bytes) 2024-05-08T08:25:42.962 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:42.962 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:42.962 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:42.962 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:42.962 INFO:teuthology.orchestra.run.smithi031.stdout:3619 read 0x765bd98 thru 0x766231b (0x6584 bytes) 2024-05-08T08:25:42.962 INFO:teuthology.orchestra.run.smithi031.stdout:3620 read 0x25b8092 thru 0x25c0143 (0x80b2 bytes) 2024-05-08T08:25:42.962 INFO:teuthology.orchestra.run.smithi031.stdout:3621 writesame 0x5243e78 thru 0x524aa70 data_size 0x1(0x6bf9 bytes) 2024-05-08T08:25:42.972 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:42.973 INFO:teuthology.orchestra.run.smithi031.stdout:3622 read 0x7966cd3 thru 0x7975ed3 (0xf201 bytes) 2024-05-08T08:25:42.973 INFO:teuthology.orchestra.run.smithi031.stdout:3623 punch from 0x91a84ef to 0x91b4508, (0xc019 bytes) 2024-05-08T08:25:42.974 INFO:teuthology.orchestra.run.smithi031.stdout:3624 trunc from 0xc712428 to 0xbb86c16 2024-05-08T08:25:42.980 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:42.980 INFO:teuthology.orchestra.run.smithi031.stdout:3626 read 0x1d10d85 thru 0x1d131e1 (0x245d bytes) 2024-05-08T08:25:42.981 INFO:teuthology.orchestra.run.smithi031.stdout:3627 punch from 0x8e9172c to 0x8e97b90, (0x6464 bytes) 2024-05-08T08:25:42.981 INFO:teuthology.orchestra.run.smithi031.stdout:3628 read 0xafba0e7 thru 0xafbfd8a (0x5ca4 bytes) 2024-05-08T08:25:42.989 INFO:teuthology.orchestra.run.smithi031.stdout:3629 trunc from 0xbb86c16 to 0x42f1492 2024-05-08T08:25:43.012 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:43.020+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:43.017 INFO:teuthology.orchestra.run.smithi031.stdout:3630 punch from 0x3b6855d to 0x3b729a7, (0xa44a bytes) 2024-05-08T08:25:43.017 INFO:teuthology.orchestra.run.smithi031.stdout:3631 read 0x2812615 thru 0x2821dd6 (0xf7c2 bytes) 2024-05-08T08:25:43.034 INFO:teuthology.orchestra.run.smithi031.stdout:3632 write 0xd248539 thru 0xd251b62 (0x962a bytes) 2024-05-08T08:25:43.038 INFO:teuthology.orchestra.run.smithi031.stdout:3633 write 0xa3c027d thru 0xa3cedae (0xeb32 bytes) 2024-05-08T08:25:43.042 INFO:teuthology.orchestra.run.smithi031.stdout:3634 writesame 0x584db54 thru 0x5853ba8 data_size 0xd(0x6055 bytes) 2024-05-08T08:25:43.046 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:43.046 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:43.046 INFO:teuthology.orchestra.run.smithi031.stdout:3638 punch from 0x565f54e to 0x566a03f, (0xaaf1 bytes) 2024-05-08T08:25:43.047 INFO:teuthology.orchestra.run.smithi031.stdout:3639 punch from 0x10a9e60 to 0x10b0980, (0x6b20 bytes) 2024-05-08T08:25:43.047 INFO:teuthology.orchestra.run.smithi031.stdout:3640 read 0x79f232b thru 0x79f9c7e (0x7954 bytes) 2024-05-08T08:25:43.047 INFO:teuthology.orchestra.run.smithi031.stdout:3641 read 0xb98abaf thru 0xb98aced (0x13f bytes) 2024-05-08T08:25:43.047 INFO:teuthology.orchestra.run.smithi031.stdout:3642 write 0x238df52 thru 0x239c78f (0xe83e bytes) 2024-05-08T08:25:43.052 INFO:teuthology.orchestra.run.smithi031.stdout:3644 read 0x196cf62 thru 0x197b3cf (0xe46e bytes) 2024-05-08T08:25:43.053 INFO:teuthology.orchestra.run.smithi031.stdout:3645 read 0x9855ae7 thru 0x985a4ff (0x4a19 bytes) 2024-05-08T08:25:43.053 INFO:teuthology.orchestra.run.smithi031.stdout:3646 writesame 0x5a14982 thru 0x5a20abc data_size 0xb(0xc13b bytes) 2024-05-08T08:25:43.058 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:43.060 INFO:teuthology.orchestra.run.smithi031.stdout:3649 trunc from 0xd251b63 to 0xb7b9573 2024-05-08T08:25:43.071 INFO:teuthology.orchestra.run.smithi031.stdout:3650 read 0x37a5c38 thru 0x37afbe5 (0x9fae bytes) 2024-05-08T08:25:43.072 INFO:teuthology.orchestra.run.smithi031.stdout:3651 read 0x9da382a thru 0x9da9ca8 (0x647f bytes) 2024-05-08T08:25:43.072 INFO:teuthology.orchestra.run.smithi031.stdout:3652 writesame 0xa887b00 thru 0xa88be65 data_size 0x1(0x4366 bytes) 2024-05-08T08:25:43.082 INFO:teuthology.orchestra.run.smithi031.stdout:3655 write 0xa6698fb thru 0xa671816 (0x7f1c bytes) 2024-05-08T08:25:43.084 INFO:teuthology.orchestra.run.smithi031.stdout:3658 read 0x1363818 thru 0x1365981 (0x216a bytes) 2024-05-08T08:25:43.085 INFO:teuthology.orchestra.run.smithi031.stdout:3659 write 0x7bd333e thru 0x7bdb3ef (0x80b2 bytes) 2024-05-08T08:25:43.090 INFO:teuthology.orchestra.run.smithi031.stdout:3661 writesame 0x19ade2f thru 0x19bd613 data_size 0x1(0xf7e5 bytes) 2024-05-08T08:25:43.096 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:43.103+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:43.111 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:43.111 INFO:teuthology.orchestra.run.smithi031.stdout:3671 writesame 0x83d92eb thru 0x83dd657 data_size 0x1(0x436d bytes) 2024-05-08T08:25:43.113 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:25:43.123 INFO:teuthology.orchestra.run.smithi031.stdout:3672 read 0xa2c6416 thru 0xa2d216a (0xbd55 bytes) 2024-05-08T08:25:43.123 INFO:teuthology.orchestra.run.smithi031.stdout:3673 trunc from 0xb7b9573 to 0xc492792 2024-05-08T08:25:43.130 INFO:teuthology.orchestra.run.smithi031.stdout:3674 punch from 0x8711c39 to 0x871e291, (0xc658 bytes) 2024-05-08T08:25:43.130 INFO:teuthology.orchestra.run.smithi031.stdout:3675 punch from 0x2a181aa to 0x2a239a3, (0xb7f9 bytes) 2024-05-08T08:25:43.130 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:43.136+0000 7f0053c1f640 -1 osd.1 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:25:43.136 INFO:teuthology.orchestra.run.smithi031.stdout:3677 write 0xd0af335 thru 0xd0b9b48 (0xa814 bytes) 2024-05-08T08:25:43.136 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:43.136 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:43.136 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:43.136 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:43.136 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:43.142 INFO:teuthology.orchestra.run.smithi031.stdout:3678 clone 22 order 21 su 131072 sc 10 2024-05-08T08:25:43.147 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:43.197 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:43.204+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:43.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:43.253 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:43.253 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21267 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.74735175, 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044009312000000002, 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.254 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21267 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513028+0000", 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.598e-06 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525553+0000", 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.012524067999999999 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525577+0000", 2024-05-08T08:25:43.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4771999999999999e-05 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525605+0000", 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7577e-05 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525700+0000", 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5251e-05 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525885+0000", 2024-05-08T08:25:43.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018491199999999999 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.530129+0000", 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042443949999999998 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556920+0000", 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02679107 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556958+0000", 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7644000000000001e-05 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556998+0000", 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0216999999999998e-05 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.557036+0000", 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7808000000000001e-05 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.59955077, 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:25:43.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:25:43.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23217 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 32505856~362265] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.801930571, 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030629341000000001, 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.265 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23217 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:43.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8099999999999998e-07 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458450+0000", 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0949999999999998e-06 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458457+0000", 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4130000000000003e-06 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458461+0000", 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3440000000000003e-06 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.488227+0000", 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029765438000000002 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.488300+0000", 2024-05-08T08:25:43.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3501e-05 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.489077+0000", 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00077716899999999997 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23222 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 19922944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.80068777, 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030552035000000002, 2024-05-08T08:25:43.269 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23222 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459693+0000", 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:25:43.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459700+0000", 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1210000000000003e-06 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459704+0000", 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4839999999999998e-06 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490142+0000", 2024-05-08T08:25:43.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030438302 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490186+0000", 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3656999999999999e-05 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490243+0000", 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6891000000000001e-05 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.273 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23223 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 24117248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.800668224000001, 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030732103, 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23223 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459711+0000", 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459716+0000", 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2290000000000002e-06 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459730+0000", 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3947999999999999e-05 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459735+0000", 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3839999999999996e-06 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490276+0000", 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030540421000000002 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490351+0000", 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5159000000000003e-05 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490442+0000", 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1607000000000006e-05 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.277 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23224 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 28311552~452228] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.800635871000001, 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031565682999999997, 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23224 2024-05-08T08:25:43.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:43.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459746+0000", 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4490000000000002e-06 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459752+0000", 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5370000000000003e-06 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459759+0000", 2024-05-08T08:25:43.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490471+0000", 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030712198999999999 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490521+0000", 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9820000000000001e-05 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491308+0000", 2024-05-08T08:25:43.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00078766700000000003 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23225 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 32505856~225280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.800613312999999, 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032237982999999998, 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.282 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23225 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459766+0000", 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.72e-07 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459769+0000", 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1200000000000002e-06 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459774+0000", 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3669999999999996e-06 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459779+0000", 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491344+0000", 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031565054000000002 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491416+0000", 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1921000000000005e-05 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492003+0000", 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058739700000000003 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23245 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 16252928~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798649789000001, 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030930742000000001, 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.286 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23245 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7800000000000001e-07 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461732+0000", 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461738+0000", 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7490000000000001e-06 2024-05-08T08:25:43.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461744+0000", 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1429999999999997e-06 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492140+0000", 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030396128000000001 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492215+0000", 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5493000000000004e-05 2024-05-08T08:25:43.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492660+0000", 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00044440799999999998 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23246 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.79862838, 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031032365999999999, 2024-05-08T08:25:43.290 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23246 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461751+0000", 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.75e-07 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461753+0000", 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461758+0000", 2024-05-08T08:25:43.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461763+0000", 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492690+0000", 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030926571999999999 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492731+0000", 2024-05-08T08:25:43.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1109999999999998e-05 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492783+0000", 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1703999999999998e-05 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23247 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 24641536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:43.294 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798609311, 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031211295999999999, 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23247 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461770+0000", 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461772+0000", 2024-05-08T08:25:43.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9640000000000002e-06 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461777+0000", 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461782+0000", 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6829999999999999e-06 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492814+0000", 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031032257000000001 2024-05-08T08:25:43.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492888+0000", 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3706000000000002e-05 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492981+0000", 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2789999999999998e-05 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23248 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.79859113, 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031307201999999999, 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23248 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461788+0000", 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461790+0000", 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2189999999999998e-06 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461796+0000", 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7690000000000002e-06 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461800+0000", 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6179999999999997e-06 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493006+0000", 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031205172 2024-05-08T08:25:43.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493044+0000", 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8244000000000002e-05 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493095+0000", 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0776000000000003e-05 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23249 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 33030144~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798572222000001, 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032220515999999998, 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23249 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461807+0000", 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1699999999999999e-07 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461809+0000", 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461814+0000", 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7780000000000004e-06 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461819+0000", 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493127+0000", 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031307802000000003 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493194+0000", 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7335000000000005e-05 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494027+0000", 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083252399999999996 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.306 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23250 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798550958, 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031361471000000002, 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23250 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:25:43.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461859+0000", 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0644e-05 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461866+0000", 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5900000000000002e-06 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461872+0000", 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:25:43.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493110+0000", 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031238542000000001 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493144+0000", 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3828999999999997e-05 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493189+0000", 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4808999999999997e-05 2024-05-08T08:25:43.310 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23252 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798493821999999, 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031396697000000001, 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.311 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23252 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.312 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7999999999999995e-07 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461888+0000", 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.869e-06 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461894+0000", 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1700000000000002e-06 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461899+0000", 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8359999999999999e-06 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493202+0000", 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031302950000000003 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493237+0000", 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5088e-05 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493281+0000", 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4203999999999999e-05 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23253 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 12058624~336650] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798472657, 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031833767999999998, 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.315 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23253 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461909+0000", 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9739999999999998e-06 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461917+0000", 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:25:43.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461921+0000", 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6909999999999996e-06 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493295+0000", 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031373511999999999 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493328+0000", 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3068000000000003e-05 2024-05-08T08:25:43.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493740+0000", 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00041189299999999998 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23254 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 16252928~115013] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798450729000001, 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032318688999999998, 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.319 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23254 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461930+0000", 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2129999999999998e-06 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461937+0000", 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4840000000000001e-06 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461941+0000", 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5589999999999999e-06 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493755+0000", 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031813266999999999 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493796+0000", 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0747999999999997e-05 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494247+0000", 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00045104900000000002 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23255 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798409969, 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032478619, 2024-05-08T08:25:43.323 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23255 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461972+0000", 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.649e-06 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461979+0000", 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0119999999999999e-06 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461984+0000", 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2580000000000003e-06 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494282+0000", 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032298329000000001 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494356+0000", 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3807999999999995e-05 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494447+0000", 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1153e-05 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23256 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 24641536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798387733, 2024-05-08T08:25:43.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032599534999999999, 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23256 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.58e-07 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461994+0000", 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3819999999999998e-06 2024-05-08T08:25:43.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462001+0000", 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4580000000000004e-06 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462006+0000", 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9599999999999999e-06 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494486+0000", 2024-05-08T08:25:43.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032479959000000003 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494530+0000", 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3825999999999997e-05 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494590+0000", 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0591999999999999e-05 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23257 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798304659999999, 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032656203000000002, 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23257 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462077+0000", 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3199999999999998e-06 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462083+0000", 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9739999999999999e-06 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462088+0000", 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3619999999999997e-06 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494622+0000", 2024-05-08T08:25:43.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032533524000000001 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494669+0000", 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7939999999999998e-05 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494730+0000", 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0639e-05 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23258 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 33030144~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.798180299, 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.033188701000000001, 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23258 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.336 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462199+0000", 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.92e-07 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462202+0000", 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462211+0000", 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4190000000000002e-06 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462217+0000", 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5570000000000001e-06 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494758+0000", 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032540628000000002 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494800+0000", 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1767999999999997e-05 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.495387+0000", 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058742199999999999 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.339 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.340 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.340 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:43.341 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:43.341 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:43.347+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:43.352 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:43.352 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:43.361 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-08T08:25:43.442 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:43.449+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:43.445 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:43.446 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:43.446 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:43.446 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:43.454 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:43.542 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:43.550+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:43.600 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:43.601 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:43.601 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:43.610 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:43.643 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:43.649+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:43.744 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:43.752+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:43.757 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1815 3.10 3:09da06bb:::rbd_data.119116cc7dd0.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 33.522666729000001, 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057340569999999999, 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.758 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1815 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241202+0000", 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241211+0000", 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6940000000000001e-06 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241228+0000", 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7487e-05 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241239+0000", 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0954999999999999e-05 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241305+0000", 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5696000000000003e-05 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241432+0000", 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000127639 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.242178+0000", 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074582499999999998 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.245608+0000", 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003430191 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246869+0000", 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001261096 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246904+0000", 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5024999999999998e-05 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246934+0000", 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9519999999999999e-05 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1992 3.a 3:525853b5:::rbd_data.119116cc7dd0.0000000000000148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:43.763 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 33.348928100999998, 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005824053, 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1992 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414940+0000", 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.029e-06 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414946+0000", 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2890000000000003e-06 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414957+0000", 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1280999999999999e-05 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414966+0000", 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415026+0000", 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0624000000000001e-05 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415093+0000", 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6752000000000001e-05 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415154+0000", 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1016e-05 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417565+0000", 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024105730000000001 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420666+0000", 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031012219999999998 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420721+0000", 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5260000000000003e-05 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420763+0000", 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1519e-05 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7101 3.a 3:570608b4:::rbd_data.119120fdbc10.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1310720~19922944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 27.548797636, 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053734150000000003, 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7101 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:43.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215071+0000", 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4210000000000002e-06 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217644+0000", 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025721239999999999 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217664+0000", 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0492999999999998e-05 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217681+0000", 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6985000000000001e-05 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217759+0000", 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7749000000000004e-05 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217900+0000", 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141361 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.218122+0000", 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022196499999999999 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220140+0000", 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020175129999999999 2024-05-08T08:25:43.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220373+0000", 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023305999999999999 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220410+0000", 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7146e-05 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220442+0000", 2024-05-08T08:25:43.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2598000000000002e-05 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24991 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3181083840000003, 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005717364, 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.774 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24991 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445759+0000", 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445760+0000", 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.796e-06 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445764+0000", 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7579999999999998e-06 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445767+0000", 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8380000000000002e-06 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448156+0000", 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023889200000000001 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448220+0000", 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4469999999999998e-05 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448567+0000", 2024-05-08T08:25:43.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00034631100000000003 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450776+0000", 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022094570000000002 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451411+0000", 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063536400000000002 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451442+0000", 2024-05-08T08:25:43.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0080999999999999e-05 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451476+0000", 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4088999999999998e-05 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24992 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3180892400000008, 2024-05-08T08:25:43.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058446770000000004, 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24992 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445778+0000", 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445780+0000", 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.367e-06 2024-05-08T08:25:43.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445785+0000", 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4830000000000001e-06 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445788+0000", 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0280000000000001e-06 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449082+0000", 2024-05-08T08:25:43.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0032946540000000002 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449148+0000", 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5945999999999995e-05 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449434+0000", 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00028535499999999999 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451491+0000", 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020569260000000002 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451531+0000", 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0187e-05 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451583+0000", 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2420999999999998e-05 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451622+0000", 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8930999999999997e-05 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24993 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3180608540000005, 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006203943, 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.785 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24993 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:43.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445809+0000", 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5000000000000002e-06 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445814+0000", 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3680000000000001e-06 2024-05-08T08:25:43.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445818+0000", 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6169999999999998e-06 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450046+0000", 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042282190000000001 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450131+0000", 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4747999999999996e-05 2024-05-08T08:25:43.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450449+0000", 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000318022 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451647+0000", 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011983479999999999 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451925+0000", 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00027754599999999999 2024-05-08T08:25:43.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451977+0000", 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2355000000000003e-05 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.452010+0000", 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2804e-05 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.790 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25148 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.1053346239999993, 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073557630000000004, 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25148 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:43.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.03e-07 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658535+0000", 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1919999999999999e-06 2024-05-08T08:25:43.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658539+0000", 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6779999999999999e-06 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658542+0000", 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.225e-06 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.659927+0000", 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013846729999999999 2024-05-08T08:25:43.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660009+0000", 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1887999999999999e-05 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660392+0000", 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00038333600000000002 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665776+0000", 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005383423 2024-05-08T08:25:43.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665836+0000", 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0297000000000002e-05 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665863+0000", 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7412000000000001e-05 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665888+0000", 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4335999999999999e-05 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.795 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25149 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.1053145779999998, 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073836789999999998, 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25149 2024-05-08T08:25:43.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:43.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7e-07 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658554+0000", 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.877e-06 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658559+0000", 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5410000000000002e-06 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658562+0000", 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8509999999999999e-06 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660808+0000", 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022458589999999998 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660858+0000", 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0250000000000002e-05 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661053+0000", 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000195269 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665807+0000", 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047536710000000001 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665894+0000", 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7678999999999999e-05 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665917+0000", 2024-05-08T08:25:43.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2921999999999998e-05 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665936+0000", 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8389999999999998e-05 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25150 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.1052962690000001, 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074116290000000003, 2024-05-08T08:25:43.801 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25150 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658571+0000", 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7099999999999998e-07 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658574+0000", 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9069999999999999e-06 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658580+0000", 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5749999999999997e-06 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658585+0000", 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999998e-06 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661498+0000", 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0029126820000000002 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661544+0000", 2024-05-08T08:25:43.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6585999999999997e-05 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661742+0000", 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019782600000000001 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665830+0000", 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040875310000000002 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665941+0000", 2024-05-08T08:25:43.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110975 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665963+0000", 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.213e-05 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665982+0000", 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9069e-05 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25151 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.1052770170000006, 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074729280000000002, 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25151 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.35e-07 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.808 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658592+0000", 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.855e-06 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658595+0000", 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7570000000000002e-06 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658598+0000", 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5900000000000002e-06 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662062+0000", 2024-05-08T08:25:43.809 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0034643909999999998 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662148+0000", 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5148999999999994e-05 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662351+0000", 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000203313 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665987+0000", 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036361060000000001 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666009+0000", 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2098000000000001e-05 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666040+0000", 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0660999999999997e-05 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666063+0000", 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2773e-05 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25289 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.798833+0000", 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9650335600000002, 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004932507, 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.812 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25289 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798833+0000", 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798833+0000", 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798834+0000", 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0559999999999999e-06 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798843+0000", 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7509999999999997e-06 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.814 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798856+0000", 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3455000000000001e-05 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798863+0000", 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9480000000000002e-06 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799425+0000", 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00056170999999999999 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799507+0000", 2024-05-08T08:25:43.815 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1588999999999994e-05 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799735+0000", 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022800199999999999 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801963+0000", 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022284079999999999 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.816 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803668+0000", 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001705018 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803721+0000", 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3227999999999999e-05 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803766+0000", 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4342000000000003e-05 2024-05-08T08:25:43.817 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25290 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9643545210000006, 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059807180000000003, 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.818 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25290 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.819 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799513+0000", 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3300000000000001e-07 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799518+0000", 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3090000000000003e-06 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799560+0000", 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1578000000000001e-05 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.820 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799568+0000", 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2849999999999995e-06 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800054+0000", 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000486121 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800137+0000", 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3003000000000005e-05 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.821 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800339+0000", 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020165600000000001 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803776+0000", 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003436691 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805429+0000", 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016535779999999999 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.822 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805461+0000", 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1674999999999997e-05 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805493+0000", 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1989000000000002e-05 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25291 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9639164450000006, 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056878800000000002, 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25291 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799951+0000", 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9100000000000003e-07 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799955+0000", 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2379999999999999e-06 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799964+0000", 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3680000000000008e-06 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799970+0000", 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9939999999999997e-06 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800828+0000", 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00085768499999999996 2024-05-08T08:25:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800877+0000", 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8609000000000002e-05 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801075+0000", 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019808400000000001 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805212+0000", 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041372850000000001 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805543+0000", 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00033049199999999999 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805595+0000", 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1919999999999998e-05 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805638+0000", 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3513999999999997e-05 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25292 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800479+0000", 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9633877359999996, 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052346249999999997, 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25292 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800479+0000", 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800479+0000", 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800480+0000", 2024-05-08T08:25:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7700000000000004e-07 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800486+0000", 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6139999999999997e-06 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800498+0000", 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1695999999999999e-05 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800505+0000", 2024-05-08T08:25:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801402+0000", 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00089720399999999997 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801484+0000", 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1305999999999997e-05 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801694+0000", 2024-05-08T08:25:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021065299999999999 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805219+0000", 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0035250070000000001 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805669+0000", 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00044995700000000002 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805695+0000", 2024-05-08T08:25:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5995000000000001e-05 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805714+0000", 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8300000000000001e-05 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25293 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9629556929999996, 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073904310000000003, 2024-05-08T08:25:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25293 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800912+0000", 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800916+0000", 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9689999999999996e-06 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800924+0000", 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3140000000000004e-06 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800930+0000", 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3450000000000002e-06 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801990+0000", 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00106022 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802073+0000", 2024-05-08T08:25:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2761000000000003e-05 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802285+0000", 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000212036 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808000+0000", 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057151379999999998 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808232+0000", 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023199399999999999 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808264+0000", 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1851999999999999e-05 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808301+0000", 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7122000000000001e-05 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25294 3.a 3:536ed5cb:::rbd_data.11911e381319.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9629216849999995, 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074925599999999997, 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25294 2024-05-08T08:25:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.77e-07 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800948+0000", 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800955+0000", 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2660000000000004e-06 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800958+0000", 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1599999999999998e-06 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802642+0000", 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001683534 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802747+0000", 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010508799999999999 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802853+0000", 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000106428 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808022+0000", 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051684570000000004 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808352+0000", 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032994200000000001 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808398+0000", 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-05 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808438+0000", 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9223e-05 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25295 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9625707919999993, 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072563819999999996, 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25295 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801297+0000", 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5300000000000003e-07 2024-05-08T08:25:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801302+0000", 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801314+0000", 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2652e-05 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801321+0000", 2024-05-08T08:25:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0899999999999999e-06 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803076+0000", 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001754802 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803156+0000", 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9699999999999999e-05 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803369+0000", 2024-05-08T08:25:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021332400000000001 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808025+0000", 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0046552039999999996 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808480+0000", 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00045548700000000003 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808515+0000", 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4782e-05 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808552+0000", 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7447999999999998e-05 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25323 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.9246343570000004, 2024-05-08T08:25:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0055662289999999998, 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25323 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839233+0000", 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839235+0000", 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:25:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839239+0000", 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839242+0000", 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9100000000000001e-06 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839832+0000", 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058972500000000004 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839911+0000", 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9762e-05 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.840110+0000", 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019887299999999999 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.841657+0000", 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001547011 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844711+0000", 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030535689999999999 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844763+0000", 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1952e-05 2024-05-08T08:25:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844799+0000", 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5627000000000003e-05 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:27262 3.10 3:0c3194f4:::rbd_data.119137f21733.000000000000020b:head [delete] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.5632967779999998, 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054741219999999997, 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 27262 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200571+0000", 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4389999999999999e-06 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200576+0000", 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200587+0000", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1209000000000001e-05 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200592+0000", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3190000000000002e-06 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200642+0000", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9870999999999997e-05 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200692+0000", 2024-05-08T08:25:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9651999999999998e-05 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200787+0000", 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5424e-05 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.202753+0000", 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0019658499999999999 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.205947+0000", 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031941119999999998 2024-05-08T08:25:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.206001+0000", 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3573999999999999e-05 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.206044+0000", 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3071999999999997e-05 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:27980 3.a 3:52e9ec1e:::rbd_data.11911e381319.0000000000000001:12 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.5741528389999999, 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051359279999999997, 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 27980 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189718+0000", 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.14e-06 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189727+0000", 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4739999999999994e-06 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189734+0000", 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0060000000000003e-06 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.194568+0000", 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048336830000000001 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.194617+0000", 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9357000000000002e-05 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.194850+0000", 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023270499999999999 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:43.866 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:43.868 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:43.868 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:43.875+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:43.968 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:43.975+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:44.013 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:44.013 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:44.013 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:44.022 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:44.069 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:44.077+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:44.118 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-05-08T08:25:44.131 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:44.132 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:44.132 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:44.132 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:44.132 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:44.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T08:25:44.142 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:44.181 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:44.187+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:44.272 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:44.279+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.730770622000001, 2024-05-08T08:25:44.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:25:44.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:25:44.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:25:44.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:88 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.575285217000001, 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084184770000000006, 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.296 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 88 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721094+0000", 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1380000000000001e-06 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721098+0000", 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4120000000000001e-06 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721100+0000", 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6910000000000002e-06 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725525+0000", 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044250560000000001 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725656+0000", 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013079000000000001 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725765+0000", 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000108571 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728094+0000", 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023291229999999998 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729456+0000", 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013616780000000001 2024-05-08T08:25:44.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729487+0000", 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0956e-05 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729510+0000", 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3711999999999999e-05 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.301 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376270923, 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084564540000000004, 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 108 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920108+0000", 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9740000000000001e-06 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920112+0000", 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920114+0000", 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6630000000000002e-06 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920996+0000", 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00088141399999999996 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921044+0000", 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8078000000000003e-05 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921120+0000", 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6469000000000003e-05 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925621+0000", 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045009630000000002 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928490+0000", 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028691929999999999 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928539+0000", 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8857000000000003e-05 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928563+0000", 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3302000000000002e-05 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.306 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376258935999999, 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085274359999999993, 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 109 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1e-07 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920144+0000", 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.586e-05 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920148+0000", 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1790000000000001e-06 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920151+0000", 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0359999999999998e-06 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921404+0000", 2024-05-08T08:25:44.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00125264 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921448+0000", 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4227000000000002e-05 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921520+0000", 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1532999999999994e-05 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925631+0000", 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041116260000000002 2024-05-08T08:25:44.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928593+0000", 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002961881 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928621+0000", 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7259000000000001e-05 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928646+0000", 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4984999999999999e-05 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.311 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376218010999999, 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085697380000000004, 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920162+0000", 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920165+0000", 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4640000000000002e-06 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920168+0000", 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6249999999999999e-06 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921849+0000", 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016816789999999999 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921895+0000", 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5198000000000003e-05 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921968+0000", 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3178999999999998e-05 2024-05-08T08:25:44.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925638+0000", 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036703859999999999 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928677+0000", 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003038303 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928708+0000", 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1186000000000003e-05 2024-05-08T08:25:44.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928729+0000", 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1035000000000001e-05 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376205196999997, 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086906299999999995, 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.317 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 111 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:44.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.03e-07 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920174+0000", 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9230000000000001e-06 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920177+0000", 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0539999999999999e-06 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920180+0000", 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8480000000000001e-06 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922200+0000", 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020199060000000001 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922246+0000", 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6319999999999997e-05 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0517000000000003e-05 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925642+0000", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033256649999999998 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928770+0000", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031275640000000002 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928829+0000", 2024-05-08T08:25:44.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8690000000000002e-05 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928862+0000", 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3939999999999997e-05 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376193350000001, 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087670669999999999, 2024-05-08T08:25:44.322 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 112 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.323 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920214+0000", 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9626999999999999e-05 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920218+0000", 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1320000000000004e-06 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920225+0000", 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5859999999999999e-06 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922574+0000", 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002349009 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922651+0000", 2024-05-08T08:25:44.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7014999999999994e-05 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922793+0000", 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141839 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928736+0000", 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005942363 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928901+0000", 2024-05-08T08:25:44.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00016587700000000001 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928930+0000", 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8487e-05 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928951+0000", 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0908e-05 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.327 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376142879, 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087948200000000001, 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 113 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:44.328 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.79e-07 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920237+0000", 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.368e-06 2024-05-08T08:25:44.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920241+0000", 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920244+0000", 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7549999999999999e-06 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923291+0000", 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030469809999999998 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923338+0000", 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6501000000000001e-05 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923409+0000", 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1198000000000005e-05 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928739+0000", 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053299819999999996 2024-05-08T08:25:44.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928985+0000", 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00024582599999999999 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929010+0000", 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5145999999999999e-05 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929029+0000", 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9184000000000001e-05 2024-05-08T08:25:44.332 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376128182999999, 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088585829999999997, 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 114 2024-05-08T08:25:44.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:44.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.11e-07 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920251+0000", 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0310000000000001e-06 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920254+0000", 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1549999999999999e-06 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920257+0000", 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7360000000000001e-06 2024-05-08T08:25:44.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923648+0000", 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033905189999999998 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923693+0000", 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170999999999998e-05 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923764+0000", 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0875000000000002e-05 2024-05-08T08:25:44.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928874+0000", 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051101599999999999 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929058+0000", 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018473 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929087+0000", 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8495000000000001e-05 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929107+0000", 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.05e-05 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376115675999998, 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089168839999999999, 2024-05-08T08:25:44.338 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 115 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920262+0000", 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920301+0000", 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9572e-05 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920305+0000", 2024-05-08T08:25:44.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2390000000000004e-06 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920309+0000", 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.14e-06 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924004+0000", 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036953369999999999 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924048+0000", 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4342999999999999e-05 2024-05-08T08:25:44.341 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924120+0000", 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1489999999999995e-05 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928881+0000", 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047611650000000004 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929135+0000", 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025369599999999998 2024-05-08T08:25:44.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929159+0000", 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4479999999999999e-05 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929178+0000", 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9208999999999999e-05 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376063049999999, 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089378329999999992, 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 116 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.344 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6199999999999999e-07 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.345 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920316+0000", 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1260000000000001e-06 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920326+0000", 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4229999999999994e-06 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920330+0000", 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:25:44.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924512+0000", 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041817440000000003 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924558+0000", 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6285000000000002e-05 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924639+0000", 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0653000000000002e-05 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928884+0000", 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042451420000000004 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929204+0000", 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032002899999999999 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929232+0000", 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8048e-05 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929252+0000", 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9581000000000001e-05 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.376040752000002, 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008977779, 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.349 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 117 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.350 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920337+0000", 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7000000000000001e-07 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920341+0000", 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3039999999999998e-06 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920346+0000", 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920348+0000", 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6809999999999998e-06 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925057+0000", 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047081240000000002 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925101+0000", 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4838e-05 2024-05-08T08:25:44.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925174+0000", 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2619000000000004e-05 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928959+0000", 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037846730000000001 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929272+0000", 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00031341099999999998 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929296+0000", 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4226000000000001e-05 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929314+0000", 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7808999999999998e-05 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:44.354 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.679988993000002, 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098144059999999995, 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 141 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3299999999999998e-07 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616391+0000", 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4360000000000001e-06 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616395+0000", 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.822e-06 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616397+0000", 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.729e-06 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622404+0000", 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0060065120000000003 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622448+0000", 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3707e-05 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622520+0000", 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624923+0000", 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024033819999999999 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626156+0000", 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001232216 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626183+0000", 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7347000000000001e-05 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626202+0000", 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9564000000000001e-05 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:6914 3.13 3:cdf44573:::rbd_data.119120fdbc10.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13546835~4324] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.140494966999999, 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0094012790000000002, 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 6914 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155884+0000", 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7060000000000001e-06 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159051+0000", 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031671109999999998 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159073+0000", 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1667000000000001e-05 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159083+0000", 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0881e-05 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159151+0000", 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7706000000000005e-05 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159252+0000", 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000101098 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159508+0000", 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025555700000000001 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.163927+0000", 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044189390000000002 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165209+0000", 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012823610000000001 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165248+0000", 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9184999999999998e-05 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165283+0000", 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5067999999999997e-05 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14412 3.d 3:b3426b33:::rbd_data.119174045df.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.272159961, 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024946593, 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.365 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14412 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024218+0000", 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024221+0000", 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5489999999999998e-06 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024231+0000", 2024-05-08T08:25:44.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.611e-06 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024236+0000", 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1950000000000002e-06 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024278+0000", 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2191000000000003e-05 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024328+0000", 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0008e-05 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.028903+0000", 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045743340000000002 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049023+0000", 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020120815 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049106+0000", 2024-05-08T08:25:44.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2900999999999998e-05 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049140+0000", 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4029000000000002e-05 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049164+0000", 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3280000000000001e-05 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23281 3.15 3:aef679f3:::rbd_data.1191ea0f1b1f.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.819926862999999, 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015244407, 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23281 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476452+0000", 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.562e-06 2024-05-08T08:25:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483043+0000", 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065909009999999997 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483062+0000", 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9538000000000002e-05 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483074+0000", 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1915e-05 2024-05-08T08:25:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483133+0000", 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9339000000000002e-05 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483271+0000", 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013750500000000001 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483684+0000", 2024-05-08T08:25:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0004126 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.486766+0000", 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030822380000000002 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491572+0000", 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048065629999999998 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491643+0000", 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0518000000000005e-05 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491695+0000", 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1727999999999997e-05 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23287 3.13 3:ced44c57:::rbd_data.1191ea0f1b1f.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.783317896, 2024-05-08T08:25:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020994202999999999, 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23287 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513061+0000", 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5889999999999999e-06 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524070+0000", 2024-05-08T08:25:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011009694 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524089+0000", 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8819000000000001e-05 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524099+0000", 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9580000000000002e-06 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524157+0000", 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8106000000000002e-05 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524271+0000", 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000113231 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524698+0000", 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00042707900000000002 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533002+0000", 2024-05-08T08:25:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083041999999999994 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533940+0000", 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00093861900000000002 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534001+0000", 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0492999999999998e-05 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534053+0000", 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2414999999999999e-05 2024-05-08T08:25:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25346 3.11 3:8ca84073:::rbd_data.11911e381319.00000000000000fb:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.4111805210000004, 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014718205, 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25346 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3799999999999996e-07 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885201+0000", 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0690000000000003e-06 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885211+0000", 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0023999999999999e-05 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885217+0000", 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4129999999999997e-06 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885258+0000", 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1616000000000003e-05 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885334+0000", 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5841999999999997e-05 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.887923+0000", 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025888949999999999 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899794+0000", 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011870500000000001 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899844+0000", 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488000000000002e-05 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899881+0000", 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6538999999999997e-05 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899915+0000", 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3980999999999999e-05 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:30455 3.15 3:ae439c8f:::rbd_data.119198308290.00000000000000e9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:42.612827+0000", 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 1.6835502180000002, 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089794030000000004, 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 30455 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612827+0000", 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612827+0000", 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612828+0000", 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6050000000000001e-06 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612835+0000", 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2079999999999997e-06 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612850+0000", 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5158e-05 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612857+0000", 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1110000000000002e-06 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612912+0000", 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4719000000000002e-05 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.613020+0000", 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010835799999999999 2024-05-08T08:25:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.614557+0000", 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0015366959999999999 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621644+0000", 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0070875410000000002 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621693+0000", 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8925999999999999e-05 2024-05-08T08:25:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621759+0000", 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5871999999999995e-05 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621806+0000", 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7209000000000002e-05 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:31360 3.f 3:f2e41b89:::rbd_data.119198308290.0000000000000036:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 1.1971809659999999, 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020342331000000002, 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 31360 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099197+0000", 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.085e-06 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099202+0000", 2024-05-08T08:25:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.78e-06 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099214+0000", 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2369e-05 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099223+0000", 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6649999999999992e-06 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099283+0000", 2024-05-08T08:25:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9784000000000002e-05 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099366+0000", 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3091999999999997e-05 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.103600+0000", 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042338469999999998 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119405+0000", 2024-05-08T08:25:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015805146999999999 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119423+0000", 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8283000000000001e-05 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119482+0000", 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9206999999999997e-05 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119538+0000", 2024-05-08T08:25:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6072000000000001e-05 2024-05-08T08:25:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:44.398 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:44.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:44.400 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:44.405+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:44.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:44.500 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:44.508+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:44.510 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:44.510 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:44.511 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:44.511 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:44.520 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:44.593 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:44.593 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:44.601 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:44.607+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:44.612 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:44.612 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-05-08T08:25:44.630 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:44.630 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:44.633 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:44.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:44.647 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:44.648 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:44.665 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:44.665 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:44.683 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:44.683 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:44.701 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:44.701 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:44.702 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:44.709+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:44.719 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:44.720 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:44.759 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:44.759 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:44.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:44.809 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:44.816+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:44.903 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:44.910+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:44.917 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:44.917 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:44.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:45.010 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:45.017+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:45.076 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:45.076 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:45.084 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:45.105 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:45.112+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:45.124 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T08:25:45.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:25:45.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T08:25:45.205 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:45.213+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:45.239 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:45.240 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:45.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:45.333 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:45.340+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:45.405 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:45.405 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:45.407 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:45.415+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:45.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:45.508 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:45.514+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:45.564 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:45.564 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:45.572 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:45.614 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:45.621+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:45.710 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:45.717+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:45.723 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:45.723 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:45.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:45.816 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:45.824+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:45.882 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:45.882 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:45.891 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:45.912 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:45.920+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:46.013 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:46.020+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:46.040 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:46.040 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:46.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:46.115 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:46.122+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:46.126 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:25:46.126 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T08:25:46.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-08T08:25:46.203 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:46.203 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:46.213 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:46.215 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:46.223+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:46.316 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:46.324+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:46.317 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone21 intact 2024-05-08T08:25:46.339 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:46.339 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:46.349 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:46.417 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:46.423+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:46.491 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-05-08T08:25:46.499 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:46.499 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:46.509 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:46.552 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:46.560+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:46.618 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:46.625+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:46.631 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:46.632 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:46.640 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:46.678 INFO:teuthology.orchestra.run.smithi031.stdout:3680 write 0x3ea30be thru 0x3ea9ff0 (0x6f33 bytes) 2024-05-08T08:25:46.689 INFO:teuthology.orchestra.run.smithi031.stdout:3681 read 0xb559b69 thru 0xb55a590 (0xa28 bytes) 2024-05-08T08:25:46.689 INFO:teuthology.orchestra.run.smithi031.stdout:3683 read 0x1608ff6 thru 0x1615a5b (0xca66 bytes) 2024-05-08T08:25:46.689 INFO:teuthology.orchestra.run.smithi031.stdout:3684 writesame 0xb24fcc7 thru 0xb25a5b4 data_size 0x7(0xa8ee bytes) 2024-05-08T08:25:46.695 INFO:teuthology.orchestra.run.smithi031.stdout:3686 write 0xbfeea49 thru 0xbffca02 (0xdfba bytes) 2024-05-08T08:25:46.706 INFO:teuthology.orchestra.run.smithi031.stdout:3687 write 0xe43775c thru 0xe443f34 (0xc7d9 bytes) 2024-05-08T08:25:46.709 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:46.710 INFO:teuthology.orchestra.run.smithi031.stdout:3690 write 0x1ee879f thru 0x1eec4cd (0x3d2f bytes) 2024-05-08T08:25:46.719 INFO:teuthology.orchestra.run.smithi031.stdout:3691 read 0xe336da9 thru 0xe341361 (0xa5b9 bytes) 2024-05-08T08:25:46.719 INFO:teuthology.orchestra.run.smithi031.stdout:3692 write 0xdb839d7 thru 0xdb8aaf3 (0x711d bytes) 2024-05-08T08:25:46.723 INFO:teuthology.orchestra.run.smithi031.stdout:3693 write 0x790404c thru 0x790f985 (0xb93a bytes) 2024-05-08T08:25:46.724 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:46.732+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:46.727 INFO:teuthology.orchestra.run.smithi031.stdout:3695 punch from 0x3a0b56d to 0x3a12f9d, (0x7a30 bytes) 2024-05-08T08:25:46.727 INFO:teuthology.orchestra.run.smithi031.stdout:3696 read 0x86b3a93 thru 0x86b8f9f (0x550d bytes) 2024-05-08T08:25:46.732 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:46.735 INFO:teuthology.orchestra.run.smithi031.stdout:3698 trunc from 0xe443f35 to 0x7301d67 2024-05-08T08:25:46.763 INFO:teuthology.orchestra.run.smithi031.stdout:3699 clone 23 order 21 su 131072 sc 9 2024-05-08T08:25:46.796 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:46.796 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:46.805 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:46.845 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:46.853+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:46.862 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:46.862 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:46.880 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:46.880 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:46.897 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:46.898 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:46.914 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:46.915 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:46.921 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:46.928+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:46.932 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:46.932 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:46.949 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:46.950 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:46.950 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:46.950 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:46.951 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:46.951 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:46.951 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:46.960 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:46.967 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:46.967 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:46.985 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:46.985 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:47.038 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:47.045+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:47.067 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:47.067 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:47.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:47.068 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:47.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.068 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21267 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:47.068 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:47.068 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.560128959, 2024-05-08T08:25:47.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044009312000000002, 2024-05-08T08:25:47.069 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.069 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.069 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.069 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.069 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.069 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.070 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21267 2024-05-08T08:25:47.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.070 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.070 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:47.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.071 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.071 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.071 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513027+0000", 2024-05-08T08:25:47.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.513028+0000", 2024-05-08T08:25:47.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.598e-06 2024-05-08T08:25:47.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525553+0000", 2024-05-08T08:25:47.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.012524067999999999 2024-05-08T08:25:47.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525577+0000", 2024-05-08T08:25:47.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4771999999999999e-05 2024-05-08T08:25:47.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525605+0000", 2024-05-08T08:25:47.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7577e-05 2024-05-08T08:25:47.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.075 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.075 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.075 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525700+0000", 2024-05-08T08:25:47.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5251e-05 2024-05-08T08:25:47.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.076 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.525885+0000", 2024-05-08T08:25:47.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018491199999999999 2024-05-08T08:25:47.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.530129+0000", 2024-05-08T08:25:47.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042443949999999998 2024-05-08T08:25:47.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556920+0000", 2024-05-08T08:25:47.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02679107 2024-05-08T08:25:47.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556958+0000", 2024-05-08T08:25:47.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7644000000000001e-05 2024-05-08T08:25:47.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.556998+0000", 2024-05-08T08:25:47.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0216999999999998e-05 2024-05-08T08:25:47.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.557036+0000", 2024-05-08T08:25:47.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7808000000000001e-05 2024-05-08T08:25:47.080 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.080 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.080 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.081 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:47.081 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:47.081 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.412327979000001, 2024-05-08T08:25:47.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:25:47.082 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.082 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.082 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.082 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.082 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.082 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.083 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:25:47.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.083 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:47.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:25:47.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:25:47.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:25:47.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:25:47.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:25:47.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:25:47.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:25:47.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:25:47.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:25:47.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.090 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:25:47.091 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23217 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 32505856~362265] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.61470778, 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030629341000000001, 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.092 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23217 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458448+0000", 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8099999999999998e-07 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458450+0000", 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0949999999999998e-06 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458457+0000", 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4130000000000003e-06 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.458461+0000", 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3440000000000003e-06 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.488227+0000", 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029765438000000002 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.488300+0000", 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3501e-05 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.489077+0000", 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00077716899999999997 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23222 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 19922944~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.613464979, 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030552035000000002, 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.096 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23222 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459691+0000", 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459693+0000", 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459700+0000", 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1210000000000003e-06 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459704+0000", 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4839999999999998e-06 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490142+0000", 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030438302 2024-05-08T08:25:47.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490186+0000", 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3656999999999999e-05 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490243+0000", 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6891000000000001e-05 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.100 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23223 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 24117248~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.613445432999999, 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030732103, 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23223 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459710+0000", 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459711+0000", 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459716+0000", 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2290000000000002e-06 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459730+0000", 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3947999999999999e-05 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459735+0000", 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3839999999999996e-06 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490276+0000", 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030540421000000002 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490351+0000", 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5159000000000003e-05 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490442+0000", 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1607000000000006e-05 2024-05-08T08:25:47.104 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23224 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 28311552~452228] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.613413080000001, 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031565682999999997, 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.105 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23224 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459743+0000", 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459746+0000", 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4490000000000002e-06 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459752+0000", 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5370000000000003e-06 2024-05-08T08:25:47.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459759+0000", 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490471+0000", 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030712198999999999 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.490521+0000", 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9820000000000001e-05 2024-05-08T08:25:47.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491308+0000", 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00078766700000000003 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23225 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 32505856~225280] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.613390522, 2024-05-08T08:25:47.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032237982999999998, 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23225 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:47.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459765+0000", 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459766+0000", 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.72e-07 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459769+0000", 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1200000000000002e-06 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459774+0000", 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3669999999999996e-06 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.459779+0000", 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491344+0000", 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031565054000000002 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491416+0000", 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1921000000000005e-05 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492003+0000", 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058739700000000003 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.113 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23245 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 16252928~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611426997999999, 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030930742000000001, 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.114 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23245 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461729+0000", 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7800000000000001e-07 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461732+0000", 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461738+0000", 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7490000000000001e-06 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461744+0000", 2024-05-08T08:25:47.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1429999999999997e-06 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492140+0000", 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030396128000000001 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492215+0000", 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5493000000000004e-05 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492660+0000", 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00044440799999999998 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23246 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:47.118 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611405589, 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031032365999999999, 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23246 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:47.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461750+0000", 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461751+0000", 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.75e-07 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461753+0000", 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1969999999999999e-06 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461758+0000", 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461763+0000", 2024-05-08T08:25:47.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7090000000000001e-06 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492690+0000", 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.030926571999999999 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492731+0000", 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1109999999999998e-05 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.122 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492783+0000", 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1703999999999998e-05 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23247 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 24641536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.61138652, 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031211295999999999, 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.123 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23247 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461769+0000", 2024-05-08T08:25:47.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461770+0000", 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3999999999999997e-07 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461772+0000", 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9640000000000002e-06 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461777+0000", 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5559999999999998e-06 2024-05-08T08:25:47.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461782+0000", 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6829999999999999e-06 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492814+0000", 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031032257000000001 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492888+0000", 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3706000000000002e-05 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.492981+0000", 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2789999999999998e-05 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23248 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.127 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611368338999998, 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031307201999999999, 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23248 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461787+0000", 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461788+0000", 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0400000000000002e-07 2024-05-08T08:25:47.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461790+0000", 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2189999999999998e-06 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461796+0000", 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7690000000000002e-06 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461800+0000", 2024-05-08T08:25:47.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6179999999999997e-06 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493006+0000", 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031205172 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493044+0000", 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8244000000000002e-05 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493095+0000", 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0776000000000003e-05 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23249 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:f [sparse-read 33030144~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611349431000001, 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032220515999999998, 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.132 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23249 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461806+0000", 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461807+0000", 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1699999999999999e-07 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461809+0000", 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8870000000000001e-06 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461814+0000", 2024-05-08T08:25:47.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7780000000000004e-06 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461819+0000", 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8729999999999998e-06 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493127+0000", 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031307802000000003 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493194+0000", 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7335000000000005e-05 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494027+0000", 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00083252399999999996 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23250 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 3670016~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611328167, 2024-05-08T08:25:47.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031361471000000002, 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23250 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.137 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:47.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.139 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:47.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461828+0000", 2024-05-08T08:25:47.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:25:47.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461859+0000", 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0644e-05 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461866+0000", 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5900000000000002e-06 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461872+0000", 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:25:47.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493110+0000", 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031238542000000001 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493144+0000", 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3828999999999997e-05 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493189+0000", 2024-05-08T08:25:47.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4808999999999997e-05 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23252 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 7864320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611271031000001, 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031396697000000001, 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.143 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23252 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461885+0000", 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7999999999999995e-07 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461888+0000", 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.869e-06 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461894+0000", 2024-05-08T08:25:47.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1700000000000002e-06 2024-05-08T08:25:47.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461899+0000", 2024-05-08T08:25:47.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8359999999999999e-06 2024-05-08T08:25:47.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493202+0000", 2024-05-08T08:25:47.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031302950000000003 2024-05-08T08:25:47.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493237+0000", 2024-05-08T08:25:47.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5088e-05 2024-05-08T08:25:47.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493281+0000", 2024-05-08T08:25:47.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4203999999999999e-05 2024-05-08T08:25:47.149 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.149 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.149 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.150 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23253 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 12058624~336650] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.150 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:47.150 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611249866000001, 2024-05-08T08:25:47.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031833767999999998, 2024-05-08T08:25:47.150 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.151 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.151 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.151 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.151 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.151 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.151 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23253 2024-05-08T08:25:47.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.152 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:47.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:47.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461906+0000", 2024-05-08T08:25:47.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7399999999999999e-07 2024-05-08T08:25:47.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461909+0000", 2024-05-08T08:25:47.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9739999999999998e-06 2024-05-08T08:25:47.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461917+0000", 2024-05-08T08:25:47.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2559999999999996e-06 2024-05-08T08:25:47.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461921+0000", 2024-05-08T08:25:47.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6909999999999996e-06 2024-05-08T08:25:47.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493295+0000", 2024-05-08T08:25:47.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031373511999999999 2024-05-08T08:25:47.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493328+0000", 2024-05-08T08:25:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3068000000000003e-05 2024-05-08T08:25:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493740+0000", 2024-05-08T08:25:47.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00041189299999999998 2024-05-08T08:25:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.159 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23254 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 16252928~115013] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611227937999999, 2024-05-08T08:25:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032318688999999998, 2024-05-08T08:25:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.160 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.161 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.161 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.161 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23254 2024-05-08T08:25:47.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.161 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.162 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:47.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.162 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.162 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:47.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461928+0000", 2024-05-08T08:25:47.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:25:47.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461930+0000", 2024-05-08T08:25:47.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2129999999999998e-06 2024-05-08T08:25:47.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461937+0000", 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4840000000000001e-06 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461941+0000", 2024-05-08T08:25:47.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5589999999999999e-06 2024-05-08T08:25:47.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.166 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.166 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.166 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493755+0000", 2024-05-08T08:25:47.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.031813266999999999 2024-05-08T08:25:47.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.493796+0000", 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0747999999999997e-05 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494247+0000", 2024-05-08T08:25:47.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00045104900000000002 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23255 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 20447232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:47.168 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611187178000002, 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032478619, 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23255 2024-05-08T08:25:47.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.170 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.170 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:47.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461969+0000", 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461972+0000", 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.649e-06 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461979+0000", 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0119999999999999e-06 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461984+0000", 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2580000000000003e-06 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494282+0000", 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032298329000000001 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494356+0000", 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3807999999999995e-05 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494447+0000", 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1153e-05 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23256 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 24641536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:47.173 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611164941999998, 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032599534999999999, 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23256 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:47.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461991+0000", 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.58e-07 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.461994+0000", 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3819999999999998e-06 2024-05-08T08:25:47.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462001+0000", 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4580000000000004e-06 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462006+0000", 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9599999999999999e-06 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494486+0000", 2024-05-08T08:25:47.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032479959000000003 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494530+0000", 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3825999999999997e-05 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494590+0000", 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0591999999999999e-05 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23257 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 28835840~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.611081869, 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032656203000000002, 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23257 2024-05-08T08:25:47.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462074+0000", 2024-05-08T08:25:47.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462077+0000", 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3199999999999998e-06 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462083+0000", 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9739999999999999e-06 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462088+0000", 2024-05-08T08:25:47.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3619999999999997e-06 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494622+0000", 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032533524000000001 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494669+0000", 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7939999999999998e-05 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494730+0000", 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0639e-05 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23258 3.1c 3:3f39b17d:::rbd_data.1191daf6073a.0000000000000001:f [sparse-read 33030144~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.610957507999998, 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.033188701000000001, 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.182 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23258 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462198+0000", 2024-05-08T08:25:47.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462199+0000", 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.92e-07 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462202+0000", 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462211+0000", 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4190000000000002e-06 2024-05-08T08:25:47.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.462217+0000", 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5570000000000001e-06 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494758+0000", 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.032540628000000002 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.494800+0000", 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1767999999999997e-05 2024-05-08T08:25:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.495387+0000", 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058742199999999999 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.186 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:47.188 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-05-08T08:25:47.188 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:47.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T08:25:47.189 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:47.195+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:47.288 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:47.295+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:47.297 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:47.297 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:47.297 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:47.297 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:47.305 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:47.389 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:47.396+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:47.413 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:47.413 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:47.413 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:47.413 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:47.413 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:47.421 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:47.490 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:47.497+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1815 3.10 3:09da06bb:::rbd_data.119116cc7dd0.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.293922774999999, 2024-05-08T08:25:47.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057340569999999999, 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1815 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:47.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241202+0000", 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241211+0000", 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6940000000000001e-06 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241228+0000", 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7487e-05 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241239+0000", 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0954999999999999e-05 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241305+0000", 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5696000000000003e-05 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241432+0000", 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000127639 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.242178+0000", 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074582499999999998 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.245608+0000", 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003430191 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246869+0000", 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001261096 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246904+0000", 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5024999999999998e-05 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246934+0000", 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9519999999999999e-05 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1992 3.a 3:525853b5:::rbd_data.119116cc7dd0.0000000000000148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.120184147000003, 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005824053, 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.536 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1992 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414940+0000", 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.029e-06 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414946+0000", 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2890000000000003e-06 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414957+0000", 2024-05-08T08:25:47.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1280999999999999e-05 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414966+0000", 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415026+0000", 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0624000000000001e-05 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415093+0000", 2024-05-08T08:25:47.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6752000000000001e-05 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415154+0000", 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1016e-05 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417565+0000", 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024105730000000001 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420666+0000", 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031012219999999998 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420721+0000", 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5260000000000003e-05 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420763+0000", 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1519e-05 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7101 3.a 3:570608b4:::rbd_data.119120fdbc10.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1310720~19922944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.320053682000001, 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053734150000000003, 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.542 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7101 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215071+0000", 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4210000000000002e-06 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217644+0000", 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025721239999999999 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217664+0000", 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0492999999999998e-05 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217681+0000", 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6985000000000001e-05 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217759+0000", 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7749000000000004e-05 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217900+0000", 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141361 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.218122+0000", 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022196499999999999 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220140+0000", 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020175129999999999 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220373+0000", 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023305999999999999 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220410+0000", 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7146e-05 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220442+0000", 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2598000000000002e-05 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24991 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:47.547 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.08936443, 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005717364, 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24991 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:47.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445759+0000", 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445760+0000", 2024-05-08T08:25:47.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.796e-06 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445764+0000", 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7579999999999998e-06 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445767+0000", 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8380000000000002e-06 2024-05-08T08:25:47.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448156+0000", 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023889200000000001 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448220+0000", 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4469999999999998e-05 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448567+0000", 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00034631100000000003 2024-05-08T08:25:47.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450776+0000", 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022094570000000002 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451411+0000", 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063536400000000002 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451442+0000", 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0080999999999999e-05 2024-05-08T08:25:47.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451476+0000", 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4088999999999998e-05 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24992 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.089345286, 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058446770000000004, 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.632 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24992 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:47.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445778+0000", 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445780+0000", 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.367e-06 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445785+0000", 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4830000000000001e-06 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445788+0000", 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0280000000000001e-06 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449082+0000", 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0032946540000000002 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449148+0000", 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5945999999999995e-05 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449434+0000", 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00028535499999999999 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451491+0000", 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020569260000000002 2024-05-08T08:25:47.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451531+0000", 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0187e-05 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451583+0000", 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2420999999999998e-05 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451622+0000", 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8930999999999997e-05 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.637 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24993 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.0893169, 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006203943, 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24993 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445809+0000", 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5000000000000002e-06 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445814+0000", 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3680000000000001e-06 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445818+0000", 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6169999999999998e-06 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450046+0000", 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042282190000000001 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450131+0000", 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4747999999999996e-05 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450449+0000", 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000318022 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451647+0000", 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011983479999999999 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451925+0000", 2024-05-08T08:25:47.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00027754599999999999 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451977+0000", 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2355000000000003e-05 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.452010+0000", 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2804e-05 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25148 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.876590670000001, 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073557630000000004, 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25148 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.03e-07 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658535+0000", 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1919999999999999e-06 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658539+0000", 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6779999999999999e-06 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658542+0000", 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.225e-06 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.659927+0000", 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013846729999999999 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660009+0000", 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1887999999999999e-05 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660392+0000", 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00038333600000000002 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665776+0000", 2024-05-08T08:25:47.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005383423 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665836+0000", 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0297000000000002e-05 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.649 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665863+0000", 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7412000000000001e-05 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665888+0000", 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4335999999999999e-05 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25149 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.650 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.876570623999999, 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073836789999999998, 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25149 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7e-07 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658554+0000", 2024-05-08T08:25:47.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.877e-06 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658559+0000", 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5410000000000002e-06 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658562+0000", 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8509999999999999e-06 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.653 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660808+0000", 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022458589999999998 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660858+0000", 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0250000000000002e-05 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661053+0000", 2024-05-08T08:25:47.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000195269 2024-05-08T08:25:47.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665807+0000", 2024-05-08T08:25:47.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047536710000000001 2024-05-08T08:25:47.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665894+0000", 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7678999999999999e-05 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665917+0000", 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2921999999999998e-05 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665936+0000", 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8389999999999998e-05 2024-05-08T08:25:47.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25150 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.876552315, 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074116290000000003, 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.657 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25150 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658571+0000", 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7099999999999998e-07 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658574+0000", 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9069999999999999e-06 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658580+0000", 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5749999999999997e-06 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658585+0000", 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999998e-06 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661498+0000", 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0029126820000000002 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661544+0000", 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6585999999999997e-05 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661742+0000", 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019782600000000001 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665830+0000", 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040875310000000002 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665941+0000", 2024-05-08T08:25:47.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110975 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665963+0000", 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.213e-05 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665982+0000", 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9069e-05 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25151 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.876533063, 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074729280000000002, 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25151 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.663 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.35e-07 2024-05-08T08:25:47.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658592+0000", 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.855e-06 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658595+0000", 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7570000000000002e-06 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658598+0000", 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5900000000000002e-06 2024-05-08T08:25:47.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662062+0000", 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0034643909999999998 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662148+0000", 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5148999999999994e-05 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662351+0000", 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000203313 2024-05-08T08:25:47.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665987+0000", 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036361060000000001 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666009+0000", 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2098000000000001e-05 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666040+0000", 2024-05-08T08:25:47.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0660999999999997e-05 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666063+0000", 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2773e-05 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25289 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.798833+0000", 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.736289606, 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.004932507, 2024-05-08T08:25:47.668 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25289 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798833+0000", 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798833+0000", 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798834+0000", 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0559999999999999e-06 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798843+0000", 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7509999999999997e-06 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798856+0000", 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3455000000000001e-05 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.798863+0000", 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9480000000000002e-06 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799425+0000", 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00056170999999999999 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799507+0000", 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1588999999999994e-05 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799735+0000", 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022800199999999999 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801963+0000", 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022284079999999999 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803668+0000", 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001705018 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803721+0000", 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3227999999999999e-05 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803766+0000", 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4342000000000003e-05 2024-05-08T08:25:47.673 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25290 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.735610567, 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059807180000000003, 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.674 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25290 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799513+0000", 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3300000000000001e-07 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799518+0000", 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3090000000000003e-06 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799560+0000", 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1578000000000001e-05 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799568+0000", 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2849999999999995e-06 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800054+0000", 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000486121 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800137+0000", 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3003000000000005e-05 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800339+0000", 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020165600000000001 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803776+0000", 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003436691 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805429+0000", 2024-05-08T08:25:47.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016535779999999999 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805461+0000", 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1674999999999997e-05 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805493+0000", 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1989000000000002e-05 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25291 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.735172491, 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056878800000000002, 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.680 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25291 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799951+0000", 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9100000000000003e-07 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799955+0000", 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2379999999999999e-06 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799964+0000", 2024-05-08T08:25:47.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3680000000000008e-06 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799970+0000", 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9939999999999997e-06 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800828+0000", 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00085768499999999996 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800877+0000", 2024-05-08T08:25:47.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8609000000000002e-05 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801075+0000", 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019808400000000001 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805212+0000", 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041372850000000001 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805543+0000", 2024-05-08T08:25:47.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00033049199999999999 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805595+0000", 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1919999999999998e-05 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805638+0000", 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3513999999999997e-05 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25292 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800479+0000", 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.734643781999999, 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0052346249999999997, 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25292 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800479+0000", 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800479+0000", 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800480+0000", 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7700000000000004e-07 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800486+0000", 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6139999999999997e-06 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800498+0000", 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1695999999999999e-05 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800505+0000", 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1160000000000001e-06 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801402+0000", 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00089720399999999997 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801484+0000", 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1305999999999997e-05 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801694+0000", 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021065299999999999 2024-05-08T08:25:47.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805219+0000", 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0035250070000000001 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805669+0000", 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00044995700000000002 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805695+0000", 2024-05-08T08:25:47.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5995000000000001e-05 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805714+0000", 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8300000000000001e-05 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25293 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.734211738999999, 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073904310000000003, 2024-05-08T08:25:47.691 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25293 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800912+0000", 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800916+0000", 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9689999999999996e-06 2024-05-08T08:25:47.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800924+0000", 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3140000000000004e-06 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800930+0000", 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3450000000000002e-06 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801990+0000", 2024-05-08T08:25:47.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00106022 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802073+0000", 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2761000000000003e-05 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802285+0000", 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000212036 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808000+0000", 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057151379999999998 2024-05-08T08:25:47.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808232+0000", 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023199399999999999 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808264+0000", 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1851999999999999e-05 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808301+0000", 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7122000000000001e-05 2024-05-08T08:25:47.696 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25294 3.a 3:536ed5cb:::rbd_data.11911e381319.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.734177730999999, 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074925599999999997, 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25294 2024-05-08T08:25:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.77e-07 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800948+0000", 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800955+0000", 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2660000000000004e-06 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800958+0000", 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1599999999999998e-06 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802642+0000", 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001683534 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802747+0000", 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010508799999999999 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802853+0000", 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000106428 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808022+0000", 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051684570000000004 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808352+0000", 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032994200000000001 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808398+0000", 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-05 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808438+0000", 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9223e-05 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25295 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.733826838000001, 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072563819999999996, 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25295 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801297+0000", 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5300000000000003e-07 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801302+0000", 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801314+0000", 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2652e-05 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801321+0000", 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0899999999999999e-06 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803076+0000", 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001754802 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803156+0000", 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9699999999999999e-05 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803369+0000", 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021332400000000001 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808025+0000", 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0046552039999999996 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808480+0000", 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00045548700000000003 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808515+0000", 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4782e-05 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808552+0000", 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7447999999999998e-05 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25323 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 12.695890403, 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0055662289999999998, 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25323 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839233+0000", 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839235+0000", 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839239+0000", 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839242+0000", 2024-05-08T08:25:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9100000000000001e-06 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839832+0000", 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058972500000000004 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839911+0000", 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9762e-05 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.840110+0000", 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019887299999999999 2024-05-08T08:25:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.841657+0000", 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001547011 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844711+0000", 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030535689999999999 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844763+0000", 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1952e-05 2024-05-08T08:25:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844799+0000", 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5627000000000003e-05 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:27262 3.10 3:0c3194f4:::rbd_data.119137f21733.000000000000020b:head [delete] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3345528239999993, 2024-05-08T08:25:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054741219999999997, 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 27262 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200571+0000", 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4389999999999999e-06 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200576+0000", 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200587+0000", 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1209000000000001e-05 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200592+0000", 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3190000000000002e-06 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200642+0000", 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9870999999999997e-05 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200692+0000", 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9651999999999998e-05 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200787+0000", 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5424e-05 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.202753+0000", 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0019658499999999999 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.205947+0000", 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031941119999999998 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.206001+0000", 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3573999999999999e-05 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.206044+0000", 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3071999999999997e-05 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:27980 3.a 3:52e9ec1e:::rbd_data.11911e381319.0000000000000001:12 [sparse-read 458752~65536] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e49)", 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 8.3454088849999994, 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051359279999999997, 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 27980 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189714+0000", 2024-05-08T08:25:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189718+0000", 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.14e-06 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189727+0000", 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4739999999999994e-06 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.189734+0000", 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0060000000000003e-06 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.194568+0000", 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048336830000000001 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.194617+0000", 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9357000000000002e-05 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:39.194850+0000", 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023270499999999999 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:47.724 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:47.725 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:47.726 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:47.733+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:47.826 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:47.834+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:47.831 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:47.831 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:47.831 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:47.832 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:47.840 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:47.928 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:47.935+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:47.944 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:47.944 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:47.944 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:47.944 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:47.944 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:47.954 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:48.028 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:48.036+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:48.062 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.502241198999997, 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.063 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:25:48.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:25:48.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:48.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:88 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.346755794000003, 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084184770000000006, 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.069 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 88 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721092+0000", 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721094+0000", 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1380000000000001e-06 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.071 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721098+0000", 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4120000000000001e-06 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.721100+0000", 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6910000000000002e-06 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725525+0000", 2024-05-08T08:25:48.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044250560000000001 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725656+0000", 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013079000000000001 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.725765+0000", 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000108571 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.728094+0000", 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023291229999999998 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729456+0000", 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013616780000000001 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729487+0000", 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0956e-05 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.729510+0000", 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3711999999999999e-05 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:108 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147741500000002, 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084564540000000004, 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.075 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 108 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920106+0000", 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920108+0000", 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9740000000000001e-06 2024-05-08T08:25:48.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920112+0000", 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2150000000000001e-06 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920114+0000", 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6630000000000002e-06 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920996+0000", 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00088141399999999996 2024-05-08T08:25:48.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921044+0000", 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8078000000000003e-05 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921120+0000", 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6469000000000003e-05 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925621+0000", 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045009630000000002 2024-05-08T08:25:48.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928490+0000", 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0028691929999999999 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928539+0000", 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8857000000000003e-05 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928563+0000", 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3302000000000002e-05 2024-05-08T08:25:48.080 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147729513000002, 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085274359999999993, 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.081 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 109 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920118+0000", 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1e-07 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920144+0000", 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.586e-05 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920148+0000", 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1790000000000001e-06 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920151+0000", 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0359999999999998e-06 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921404+0000", 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00125264 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921448+0000", 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4227000000000002e-05 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921520+0000", 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1532999999999994e-05 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925631+0000", 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041116260000000002 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928593+0000", 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002961881 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928621+0000", 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7259000000000001e-05 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928646+0000", 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4984999999999999e-05 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.086 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147688588000001, 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085697380000000004, 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920159+0000", 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920162+0000", 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.243e-06 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920165+0000", 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4640000000000002e-06 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920168+0000", 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6249999999999999e-06 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921849+0000", 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016816789999999999 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921895+0000", 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5198000000000003e-05 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.921968+0000", 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3178999999999998e-05 2024-05-08T08:25:48.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925638+0000", 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036703859999999999 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928677+0000", 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003038303 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928708+0000", 2024-05-08T08:25:48.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1186000000000003e-05 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928729+0000", 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1035000000000001e-05 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147675774, 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086906299999999995, 2024-05-08T08:25:48.092 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 111 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920172+0000", 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.03e-07 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920174+0000", 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9230000000000001e-06 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920177+0000", 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0539999999999999e-06 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920180+0000", 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8480000000000001e-06 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922200+0000", 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020199060000000001 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922246+0000", 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6319999999999997e-05 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922317+0000", 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0517000000000003e-05 2024-05-08T08:25:48.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925642+0000", 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033256649999999998 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928770+0000", 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031275640000000002 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928829+0000", 2024-05-08T08:25:48.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8690000000000002e-05 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928862+0000", 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3939999999999997e-05 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147663927000004, 2024-05-08T08:25:48.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087670669999999999, 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 112 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:48.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920184+0000", 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920214+0000", 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9626999999999999e-05 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920218+0000", 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1320000000000004e-06 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920225+0000", 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5859999999999999e-06 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922574+0000", 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002349009 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922651+0000", 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7014999999999994e-05 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.922793+0000", 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141839 2024-05-08T08:25:48.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928736+0000", 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005942363 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928901+0000", 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00016587700000000001 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928930+0000", 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8487e-05 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928951+0000", 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0908e-05 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147613456000002, 2024-05-08T08:25:48.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087948200000000001, 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 113 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920234+0000", 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.79e-07 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920237+0000", 2024-05-08T08:25:48.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.368e-06 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920241+0000", 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920244+0000", 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7549999999999999e-06 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923291+0000", 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030469809999999998 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923338+0000", 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6501000000000001e-05 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923409+0000", 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1198000000000005e-05 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928739+0000", 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053299819999999996 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928985+0000", 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00024582599999999999 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929010+0000", 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5145999999999999e-05 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929029+0000", 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9184000000000001e-05 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147598760000001, 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088585829999999997, 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.110 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 114 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920249+0000", 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.11e-07 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920251+0000", 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0310000000000001e-06 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920254+0000", 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1549999999999999e-06 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920257+0000", 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7360000000000001e-06 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923648+0000", 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0033905189999999998 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923693+0000", 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5170999999999998e-05 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.923764+0000", 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0875000000000002e-05 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928874+0000", 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051101599999999999 2024-05-08T08:25:48.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929058+0000", 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00018473 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929087+0000", 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8495000000000001e-05 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929107+0000", 2024-05-08T08:25:48.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.05e-05 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147586253, 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089168839999999999, 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.116 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 115 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920261+0000", 2024-05-08T08:25:48.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920262+0000", 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1299999999999999e-07 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920301+0000", 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9572e-05 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.118 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920305+0000", 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2390000000000004e-06 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920309+0000", 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.14e-06 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924004+0000", 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036953369999999999 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924048+0000", 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4342999999999999e-05 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924120+0000", 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1489999999999995e-05 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928881+0000", 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047611650000000004 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929135+0000", 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025369599999999998 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929159+0000", 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4479999999999999e-05 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929178+0000", 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9208999999999999e-05 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.121 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147533627000001, 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089378329999999992, 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 116 2024-05-08T08:25:48.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920314+0000", 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6199999999999999e-07 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920316+0000", 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1260000000000001e-06 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920326+0000", 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4229999999999994e-06 2024-05-08T08:25:48.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920330+0000", 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5399999999999997e-06 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924512+0000", 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041817440000000003 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924558+0000", 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6285000000000002e-05 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.924639+0000", 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0653000000000002e-05 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928884+0000", 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042451420000000004 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929204+0000", 2024-05-08T08:25:48.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032002899999999999 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929232+0000", 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8048e-05 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929252+0000", 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9581000000000001e-05 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 50.147511328999997, 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.008977779, 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 117 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920336+0000", 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920337+0000", 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7000000000000001e-07 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920341+0000", 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3039999999999998e-06 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920346+0000", 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8239999999999999e-06 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.920348+0000", 2024-05-08T08:25:48.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6809999999999998e-06 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925057+0000", 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047081240000000002 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925101+0000", 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4838e-05 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.925174+0000", 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2619000000000004e-05 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.928959+0000", 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037846730000000001 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929272+0000", 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00031341099999999998 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929296+0000", 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4226000000000001e-05 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.929314+0000", 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7808999999999998e-05 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 49.451459569999997, 2024-05-08T08:25:48.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098144059999999995, 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 141 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616388+0000", 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3299999999999998e-07 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616391+0000", 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4360000000000001e-06 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616395+0000", 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.822e-06 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.616397+0000", 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.729e-06 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622404+0000", 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0060065120000000003 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622448+0000", 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3707e-05 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.622520+0000", 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2358000000000001e-05 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.624923+0000", 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024033819999999999 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626156+0000", 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001232216 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626183+0000", 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7347000000000001e-05 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:58.626202+0000", 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9564000000000001e-05 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:6914 3.13 3:cdf44573:::rbd_data.119120fdbc10.0000000000000004:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write 13546835~4324] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.911965544000001, 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0094012790000000002, 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 6914 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.139 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155882+0000", 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.155884+0000", 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7060000000000001e-06 2024-05-08T08:25:48.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159051+0000", 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031671109999999998 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159073+0000", 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1667000000000001e-05 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159083+0000", 2024-05-08T08:25:48.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0881e-05 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159151+0000", 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7706000000000005e-05 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159252+0000", 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000101098 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:48.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.159508+0000", 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00025555700000000001 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.163927+0000", 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044189390000000002 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165209+0000", 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012823610000000001 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165248+0000", 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9184999999999998e-05 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.165283+0000", 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5067999999999997e-05 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14412 3.d 3:b3426b33:::rbd_data.119174045df.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 23.043630537999999, 2024-05-08T08:25:48.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024946593, 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14412 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024218+0000", 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024221+0000", 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5489999999999998e-06 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024231+0000", 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.611e-06 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024236+0000", 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1950000000000002e-06 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024278+0000", 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2191000000000003e-05 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024328+0000", 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0008e-05 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.028903+0000", 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045743340000000002 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049023+0000", 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020120815 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049106+0000", 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2900999999999998e-05 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049140+0000", 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4029000000000002e-05 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049164+0000", 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3280000000000001e-05 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23281 3.15 3:aef679f3:::rbd_data.1191ea0f1b1f.0000000000000003:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.591397440000001, 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015244407, 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23281 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476450+0000", 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.476452+0000", 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.562e-06 2024-05-08T08:25:48.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483043+0000", 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0065909009999999997 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483062+0000", 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9538000000000002e-05 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483074+0000", 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1915e-05 2024-05-08T08:25:48.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483133+0000", 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9339000000000002e-05 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483271+0000", 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00013750500000000001 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.483684+0000", 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0004126 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.486766+0000", 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030822380000000002 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491572+0000", 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0048065629999999998 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491643+0000", 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0518000000000005e-05 2024-05-08T08:25:48.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.491695+0000", 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1727999999999997e-05 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23287 3.13 3:ced44c57:::rbd_data.1191ea0f1b1f.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.554788472999999, 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020994202999999999, 2024-05-08T08:25:48.155 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23287 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513061+0000", 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5889999999999999e-06 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524070+0000", 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011009694 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524089+0000", 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8819000000000001e-05 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524099+0000", 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9580000000000002e-06 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524157+0000", 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8106000000000002e-05 2024-05-08T08:25:48.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524271+0000", 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000113231 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524698+0000", 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00042707900000000002 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533002+0000", 2024-05-08T08:25:48.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083041999999999994 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533940+0000", 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00093861900000000002 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534001+0000", 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0492999999999998e-05 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534053+0000", 2024-05-08T08:25:48.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2414999999999999e-05 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25346 3.11 3:8ca84073:::rbd_data.11911e381319.00000000000000fb:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.182651098000001, 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014718205, 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:48.161 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25346 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885197+0000", 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3799999999999996e-07 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885201+0000", 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0690000000000003e-06 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885211+0000", 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0023999999999999e-05 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885217+0000", 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4129999999999997e-06 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885258+0000", 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1616000000000003e-05 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.885334+0000", 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5841999999999997e-05 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.887923+0000", 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025888949999999999 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899794+0000", 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011870500000000001 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899844+0000", 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488000000000002e-05 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899881+0000", 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6538999999999997e-05 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.899915+0000", 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3980999999999999e-05 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:30455 3.15 3:ae439c8f:::rbd_data.119198308290.00000000000000e9:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:42.612827+0000", 2024-05-08T08:25:48.166 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4550207950000003, 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089794030000000004, 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 30455 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612827+0000", 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612827+0000", 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612828+0000", 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6050000000000001e-06 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612835+0000", 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2079999999999997e-06 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612850+0000", 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5158e-05 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612857+0000", 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1110000000000002e-06 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.612912+0000", 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4719000000000002e-05 2024-05-08T08:25:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.613020+0000", 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010835799999999999 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.614557+0000", 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0015366959999999999 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621644+0000", 2024-05-08T08:25:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0070875410000000002 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621693+0000", 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8925999999999999e-05 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621759+0000", 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5871999999999995e-05 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:42.621806+0000", 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7209000000000002e-05 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:31360 3.f 3:f2e41b89:::rbd_data.119198308290.0000000000000036:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 4.968651543, 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020342331000000002, 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 31360 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099197+0000", 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.085e-06 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099202+0000", 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.78e-06 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099214+0000", 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2369e-05 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099223+0000", 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6649999999999992e-06 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099283+0000", 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9784000000000002e-05 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099366+0000", 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3091999999999997e-05 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.103600+0000", 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042338469999999998 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119405+0000", 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015805146999999999 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119423+0000", 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8283000000000001e-05 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119482+0000", 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9206999999999997e-05 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119538+0000", 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6072000000000001e-05 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:48.178 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:48.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:25:48.181 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T08:25:48.181 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:48.181 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:48.186+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:48.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-08T08:25:48.223 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone22 2024-05-08T08:25:48.280 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:48.287+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:48.381 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:48.388+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:48.427 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:48.427 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:48.427 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:48.443 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:48.485 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:48.492+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:48.523 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-05-08T08:25:48.583 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:48.590+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:48.643 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:48.643 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:48.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:48.684 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:48.691+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:48.785 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:48.792+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:48.819 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:48.819 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:48.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:48.819 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:48.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33198 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 0~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.237022+0000", 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58854055900000002, 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096128600000000008, 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33198 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237022+0000", 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237022+0000", 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237023+0000", 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-05-08T08:25:48.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237025+0000", 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.447e-06 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237032+0000", 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7830000000000001e-06 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237037+0000", 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.246207+0000", 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091704860000000003 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.246278+0000", 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0927999999999996e-05 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.246635+0000", 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00035701599999999998 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33200 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 65536~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.237044+0000", 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58851865400000003, 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009953567, 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.824 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33200 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237044+0000", 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237044+0000", 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237045+0000", 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9499999999999998e-07 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237047+0000", 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3769999999999999e-06 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237054+0000", 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5320000000000003e-06 2024-05-08T08:25:48.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237058+0000", 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6349999999999997e-06 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.246666+0000", 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096076519999999995 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.246740+0000", 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3920999999999999e-05 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.246998+0000", 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025805499999999998 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33202 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 98304~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.237066+0000", 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58849700900000002, 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010138135, 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33202 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237066+0000", 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237066+0000", 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237066+0000", 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237069+0000", 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3530000000000002e-06 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237075+0000", 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4019999999999999e-06 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237080+0000", 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.848e-06 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247029+0000", 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0099492260000000003 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247103+0000", 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4066000000000005e-05 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247204+0000", 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100826 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33212 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 32768~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.237088+0000", 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58847534099999999, 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010299133, 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.832 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33212 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237088+0000", 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237088+0000", 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237088+0000", 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2e-07 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237096+0000", 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.276e-06 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237103+0000", 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0899999999999999e-06 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.237109+0000", 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247233+0000", 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010124721999999999 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247301+0000", 2024-05-08T08:25:48.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7626000000000006e-05 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247387+0000", 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5822999999999995e-05 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33232 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1376256~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.238790+0000", 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58677252400000002, 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092267459999999992, 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.836 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33232 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238790+0000", 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238790+0000", 2024-05-08T08:25:48.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238791+0000", 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1099999999999996e-07 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238794+0000", 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9569999999999998e-06 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238801+0000", 2024-05-08T08:25:48.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1319999999999996e-06 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238816+0000", 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4800999999999999e-05 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247836+0000", 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.009020587 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.247909+0000", 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2504999999999998e-05 2024-05-08T08:25:48.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.248017+0000", 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000108253 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33234 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1409024~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.238825+0000", 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58673798799999999, 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094063359999999995, 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33234 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238825+0000", 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238825+0000", 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238825+0000", 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238829+0000", 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1920000000000001e-06 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238836+0000", 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.002e-06 2024-05-08T08:25:48.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238841+0000", 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1669999999999998e-06 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.248048+0000", 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0092072979999999992 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.248115+0000", 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6703000000000002e-05 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.248231+0000", 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000116506 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33243 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 131072~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.238849+0000", 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58671408899999999, 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011176762, 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33243 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238849+0000", 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238849+0000", 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238849+0000", 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1899999999999998e-07 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238858+0000", 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5499999999999995e-06 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238865+0000", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.199e-06 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238878+0000", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.311e-05 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.248261+0000", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093824609999999999 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.248330+0000", 2024-05-08T08:25:48.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9871999999999998e-05 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250026+0000", 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.001695151 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33280 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1441792~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.238892+0000", 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58667092700000001, 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011347018, 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33280 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238892+0000", 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238892+0000", 2024-05-08T08:25:48.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238892+0000", 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238896+0000", 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.106e-06 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238903+0000", 2024-05-08T08:25:48.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7400000000000004e-06 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238909+0000", 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4530000000000001e-06 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250064+0000", 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155537 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250147+0000", 2024-05-08T08:25:48.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2713000000000006e-05 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250239+0000", 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2028999999999996e-05 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33282 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1474560~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.238917+0000", 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58664609999999995, 2024-05-08T08:25:48.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011516821999999999, 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33282 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238917+0000", 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238917+0000", 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238917+0000", 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238920+0000", 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6469999999999998e-06 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238926+0000", 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3949999999999998e-06 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238931+0000", 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8289999999999998e-06 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250269+0000", 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011337583 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250338+0000", 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9282999999999994e-05 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250434+0000", 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5624000000000005e-05 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33284 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1507328~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.856 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.238939+0000", 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58662341500000004, 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011674240000000001, 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33284 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238939+0000", 2024-05-08T08:25:48.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238939+0000", 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238940+0000", 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3799999999999998e-07 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238943+0000", 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7779999999999999e-06 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238949+0000", 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5320000000000003e-06 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238954+0000", 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0189999999999997e-06 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250462+0000", 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011508116000000001 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250531+0000", 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8373000000000004e-05 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250614+0000", 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2984000000000004e-05 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.860 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33286 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1540096~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.238966+0000", 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58659725600000001, 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01182799, 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33286 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.861 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238966+0000", 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238966+0000", 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238966+0000", 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8999999999999997e-07 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238969+0000", 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.044e-06 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238976+0000", 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7719999999999997e-06 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.238981+0000", 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5040000000000002e-06 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250644+0000", 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.01166205 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250709+0000", 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5776000000000002e-05 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.250794+0000", 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4353999999999999e-05 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33321 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1146880~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.242212+0000", 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.583351124, 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093506760000000005, 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.865 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33321 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242212+0000", 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242212+0000", 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242213+0000", 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2600000000000002e-07 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242217+0000", 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7310000000000001e-06 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242225+0000", 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4889999999999998e-06 2024-05-08T08:25:48.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242230+0000", 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2190000000000003e-06 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251417+0000", 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0091872069999999993 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251479+0000", 2024-05-08T08:25:48.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1972999999999998e-05 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251562+0000", 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3331000000000006e-05 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33374 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1179648~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.242238+0000", 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58332447399999998, 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094863139999999992, 2024-05-08T08:25:48.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33374 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242238+0000", 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242238+0000", 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242239+0000", 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4899999999999995e-07 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242242+0000", 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9720000000000001e-06 2024-05-08T08:25:48.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242249+0000", 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6000000000000003e-06 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242254+0000", 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1850000000000003e-06 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251589+0000", 2024-05-08T08:25:48.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0093347039999999992 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251649+0000", 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0749999999999999e-05 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251725+0000", 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5454e-05 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33375 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1212416~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.242261+0000", 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58330201699999995, 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096524550000000008, 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33375 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242261+0000", 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242261+0000", 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242261+0000", 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:25:48.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242264+0000", 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.638e-06 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242270+0000", 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1789999999999996e-06 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242275+0000", 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2920000000000003e-06 2024-05-08T08:25:48.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251751+0000", 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0094753739999999999 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251826+0000", 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5598000000000004e-05 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251913+0000", 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6991999999999997e-05 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33376 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1245184~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.242282+0000", 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58328096600000001, 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098591200000000007, 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.878 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33376 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242282+0000", 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242282+0000", 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242282+0000", 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1e-07 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242291+0000", 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0410000000000003e-06 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242299+0000", 2024-05-08T08:25:48.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4170000000000003e-06 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242304+0000", 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4829999999999999e-06 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.251956+0000", 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0096520970000000001 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.252039+0000", 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2789999999999998e-05 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.252141+0000", 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000101982 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33377 3.19 3:983ee50e:::rbd_data.119154fa2cd3.0000000000000001:16 [read 1277952~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.242317+0000", 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.58324541699999999, 2024-05-08T08:25:48.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010032157999999999, 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33377 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242317+0000", 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242317+0000", 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242318+0000", 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9800000000000003e-07 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242321+0000", 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6850000000000001e-06 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242328+0000", 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0060000000000003e-06 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.242333+0000", 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1819999999999997e-06 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.252170+0000", 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0098370929999999999 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.252246+0000", 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5544999999999996e-05 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.252350+0000", 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010404900000000001 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33617 3.0 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.269763+0000", 2024-05-08T08:25:48.886 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.55580027099999996, 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017264730999999998, 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33617 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.269763+0000", 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.269763+0000", 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.269764+0000", 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7370000000000001e-06 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271416+0000", 2024-05-08T08:25:48.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0016517840000000001 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271431+0000", 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4600000000000001e-05 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271451+0000", 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9845999999999999e-05 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271590+0000", 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000139386 2024-05-08T08:25:48.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271766+0000", 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017567400000000001 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272039+0000", 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00027334199999999999 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.279036+0000", 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0069967340000000001 2024-05-08T08:25:48.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.286910+0000", 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078741090000000007 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.286977+0000", 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7478999999999995e-05 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.287027+0000", 2024-05-08T08:25:48.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0040000000000002e-05 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33618 3.0 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head [write 0~0] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.271480+0000", 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.55408265400000001, 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015678483, 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.892 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33618 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271480+0000", 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271480+0000", 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271481+0000", 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1829999999999999e-06 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271485+0000", 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9020000000000001e-06 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271495+0000", 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4220000000000006e-06 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.271501+0000", 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2670000000000003e-06 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.275479+0000", 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003977525 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.275570+0000", 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1207999999999998e-05 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.275745+0000", 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017509399999999999 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.279050+0000", 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033055060000000002 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.287071+0000", 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0080210560000000004 2024-05-08T08:25:48.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.287120+0000", 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8449000000000003e-05 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.287159+0000", 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8871000000000001e-05 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33693 3.19 3:9c5d0a29:::rbd_data.11915c46bf0c.0000000000000008:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.897 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.283448+0000", 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.54211453399999998, 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011652025, 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33693 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.283448+0000", 2024-05-08T08:25:48.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.283448+0000", 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.283451+0000", 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0740000000000001e-06 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284203+0000", 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00075166999999999996 2024-05-08T08:25:48.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284225+0000", 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1599000000000001e-05 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284239+0000", 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4698999999999999e-05 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284486+0000", 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024674799999999998 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284699+0000", 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000212637 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.285121+0000", 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042186900000000001 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.292965+0000", 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0078441129999999998 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.294995+0000", 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00202973 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.295055+0000", 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0254000000000003e-05 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.295100+0000", 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5631999999999999e-05 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:33694 3.19 3:9c5d0a29:::rbd_data.11915c46bf0c.0000000000000008:head [write 0~0] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.284283+0000", 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.54127982200000002, 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010943914000000001, 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 33694 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:48.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284283+0000", 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284283+0000", 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284284+0000", 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.226e-06 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284289+0000", 2024-05-08T08:25:48.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9740000000000001e-06 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284298+0000", 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9330000000000007e-06 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.284304+0000", 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0390000000000001e-06 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.289174+0000", 2024-05-08T08:25:48.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0048697640000000004 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.289254+0000", 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9633000000000001e-05 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 3", 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.289423+0000", 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000169748 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:48.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.293993+0000", 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045698429999999996 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.295143+0000", 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0011497910000000001 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.295192+0000", 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9459000000000002e-05 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:48.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.295227+0000", 2024-05-08T08:25:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4504e-05 2024-05-08T08:25:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:48.908 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:48.908 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:48.910 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:48.910 INFO:teuthology.orchestra.run.smithi031.stdout:3700 write 0xc4aae51 thru 0xc4aefd9 (0x4189 bytes) 2024-05-08T08:25:48.910 INFO:teuthology.orchestra.run.smithi031.stdout:3702 read 0xa3a1415 thru 0xa3abbb9 (0xa7a5 bytes) 2024-05-08T08:25:48.911 INFO:teuthology.orchestra.run.smithi031.stdout:3703 write 0x83d3e3e thru 0x83d7e15 (0x3fd8 bytes) 2024-05-08T08:25:48.911 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:48.911 INFO:teuthology.orchestra.run.smithi031.stdout:3705 write 0xd91ce44 thru 0xd92457b (0x7738 bytes) 2024-05-08T08:25:48.911 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:48.911 INFO:teuthology.orchestra.run.smithi031.stdout:3707 writesame 0x4eee29c thru 0x4efa853 data_size 0x72(0xc5b8 bytes) 2024-05-08T08:25:48.911 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:48.916+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:48.911 INFO:teuthology.orchestra.run.smithi031.stdout:3708 writesame 0x7fe819b thru 0x7fee255 data_size 0x1(0x60bb bytes) 2024-05-08T08:25:48.923 INFO:teuthology.orchestra.run.smithi031.stdout:3712 write 0xbd63d1c thru 0xbd6c20f (0x84f4 bytes) 2024-05-08T08:25:48.926 INFO:teuthology.orchestra.run.smithi031.stdout:3714 write 0x6b3189d thru 0x6b39cd0 (0x8434 bytes) 2024-05-08T08:25:48.934 INFO:teuthology.orchestra.run.smithi031.stdout:3715 trunc from 0xd92457c to 0xaab60ca 2024-05-08T08:25:48.945 INFO:teuthology.orchestra.run.smithi031.stdout:3717 punch from 0x5c2e755 to 0x5c355fa, (0x6ea5 bytes) 2024-05-08T08:25:48.955 INFO:teuthology.orchestra.run.smithi031.stdout:3718 writesame 0xd070757 thru 0xd07b7e3 data_size 0x1(0xb08d bytes) 2024-05-08T08:25:48.955 INFO:teuthology.orchestra.run.smithi031.stdout:3719 read 0xbaff680 thru 0xbb04380 (0x4d01 bytes) 2024-05-08T08:25:48.955 INFO:teuthology.orchestra.run.smithi031.stdout:3720 write 0xa521916 thru 0xa526e87 (0x5572 bytes) 2024-05-08T08:25:48.958 INFO:teuthology.orchestra.run.smithi031.stdout:3722 write 0x59a6eae thru 0x59b5512 (0xe665 bytes) 2024-05-08T08:25:48.963 INFO:teuthology.orchestra.run.smithi031.stdout:3724 write 0xace7ed9 thru 0xace97b7 (0x18df bytes) 2024-05-08T08:25:48.978 INFO:teuthology.orchestra.run.smithi031.stdout:3725 trunc from 0xd07b7e4 to 0x3dd8dd3 2024-05-08T08:25:48.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:49.007 INFO:teuthology.orchestra.run.smithi031.stdout:3727 read 0x1afaa5 thru 0x1bc10d (0xc669 bytes) 2024-05-08T08:25:49.008 INFO:teuthology.orchestra.run.smithi031.stdout:3729 read 0x2f6690c thru 0x2f67c08 (0x12fd bytes) 2024-05-08T08:25:49.010 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:49.018+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:3730 writesame 0x63a8b36 thru 0x63aa453 data_size 0x1(0x191e bytes) 2024-05-08T08:25:49.017 INFO:teuthology.orchestra.run.smithi031.stdout:3732 writesame 0x52d8700 thru 0x52d9e39 data_size 0x3(0x173a bytes) 2024-05-08T08:25:49.021 INFO:teuthology.orchestra.run.smithi031.stdout:3735 read 0x384d8c5 thru 0x3857020 (0x975c bytes) 2024-05-08T08:25:49.021 INFO:teuthology.orchestra.run.smithi031.stdout:3736 read 0x4ef790 thru 0x4fde91 (0xe702 bytes) 2024-05-08T08:25:49.027 INFO:teuthology.orchestra.run.smithi031.stdout:3738 trunc from 0x63aa454 to 0x2e6398e 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:37690 3.1f 3:ff9e964d:::rbd_header.11914ad0077c:head [call rbd.set_size in=19b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:49.064482+0000", 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.00020213699999999999, 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022925199999999999, 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:49.057 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 37690 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064482+0000", 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064482+0000", 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:49.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064484+0000", 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.482e-06 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064491+0000", 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1629999999999999e-06 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064507+0000", 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5897999999999999e-05 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064515+0000", 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6559999999999998e-06 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064568+0000", 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2302000000000002e-05 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:49.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:49.064626+0000", 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8752000000000003e-05 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 1 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:49.061 INFO:teuthology.orchestra.run.smithi031.stdout:3739 write 0x22331a2 thru 0x223766c (0x44cb bytes) 2024-05-08T08:25:49.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:49.085 INFO:teuthology.orchestra.run.smithi031.stdout:3740 writesame 0xa448c76 thru 0xa4560e9 data_size 0x2(0xd474 bytes) 2024-05-08T08:25:49.088 INFO:teuthology.orchestra.run.smithi031.stdout:3741 trunc from 0xa4560ea to 0xc80a2f1 2024-05-08T08:25:49.094 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.094 INFO:teuthology.orchestra.run.smithi031.stdout:3744 writesame 0x767dca5 thru 0x76808ca data_size 0x1(0x2c26 bytes) 2024-05-08T08:25:49.101 INFO:teuthology.orchestra.run.smithi031.stdout:3745 write 0x41e0a2f thru 0x41e9b09 (0x90db bytes) 2024-05-08T08:25:49.104 INFO:teuthology.orchestra.run.smithi031.stdout:3746 write 0x260e95b thru 0x261a47a (0xbb20 bytes) 2024-05-08T08:25:49.106 INFO:teuthology.orchestra.run.smithi031.stdout:3748 punch from 0xb3fa10e to 0xb3fbbff, (0x1af1 bytes) 2024-05-08T08:25:49.107 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.107 INFO:teuthology.orchestra.run.smithi031.stdout:3753 trunc from 0xc80a2f1 to 0xc8aa76e 2024-05-08T08:25:49.111 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:49.118+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:49.112 INFO:teuthology.orchestra.run.smithi031.stdout:3754 read 0x72589f8 thru 0x725a25e (0x1867 bytes) 2024-05-08T08:25:49.112 INFO:teuthology.orchestra.run.smithi031.stdout:3755 write 0xc692ab thru 0xc70c39 (0x798f bytes) 2024-05-08T08:25:49.127 INFO:teuthology.orchestra.run.smithi031.stdout:3757 trunc from 0xc8aa76e to 0xeb482be 2024-05-08T08:25:49.133 INFO:teuthology.orchestra.run.smithi031.stdout:3758 write 0x6e49ca7 thru 0x6e4a391 (0x6eb bytes) 2024-05-08T08:25:49.136 INFO:teuthology.orchestra.run.smithi031.stdout:3759 writesame 0x56199e8 thru 0x56294e5 data_size 0x3(0xfafe bytes) 2024-05-08T08:25:49.144 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:25:49.147 INFO:teuthology.orchestra.run.smithi031.stdout:3760 punch from 0xa9efb56 to 0xa9f0dc0, (0x126a bytes) 2024-05-08T08:25:49.147 INFO:teuthology.orchestra.run.smithi031.stdout:3762 write 0x812bb4c thru 0x812e234 (0x26e9 bytes) 2024-05-08T08:25:49.151 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:49.151 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:49.151 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-05-08T08:25:49.156 INFO:teuthology.orchestra.run.smithi031.stdout:3763 read 0x1065f73 thru 0x106decc (0x7f5a bytes) 2024-05-08T08:25:49.160 INFO:teuthology.orchestra.run.smithi031.stdout:3764 punch from 0xd9f126f to 0xd9f9652, (0x83e3 bytes) 2024-05-08T08:25:49.163 INFO:teuthology.orchestra.run.smithi031.stdout:3766 trunc from 0xeb482be to 0xc50f7c9 2024-05-08T08:25:49.169 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.169 INFO:teuthology.orchestra.run.smithi031.stdout:3768 read 0x334ab27 thru 0x33516be (0x6b98 bytes) 2024-05-08T08:25:49.171 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-08T08:25:49.171 INFO:teuthology.orchestra.run.smithi031.stdout:3769 write 0x5e52e6d thru 0x5e57732 (0x48c6 bytes) 2024-05-08T08:25:49.174 INFO:teuthology.orchestra.run.smithi031.stdout:3770 punch from 0x5e33601 to 0x5e375ce, (0x3fcd bytes) 2024-05-08T08:25:49.174 INFO:teuthology.orchestra.run.smithi031.stdout:3771 punch from 0xa2cb845 to 0xa2ce699, (0x2e54 bytes) 2024-05-08T08:25:49.174 INFO:teuthology.orchestra.run.smithi031.stdout:3774 punch from 0x98351a8 to 0x983537f, (0x1d7 bytes) 2024-05-08T08:25:49.174 INFO:teuthology.orchestra.run.smithi031.stdout:3776 punch from 0x8aae252 to 0x8aafcf5, (0x1aa3 bytes) 2024-05-08T08:25:49.177 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:49.177 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:49.179 INFO:teuthology.orchestra.run.smithi031.stdout:3777 trunc from 0xc50f7c9 to 0x8b71bd5 2024-05-08T08:25:49.187 INFO:teuthology.orchestra.run.smithi031.stdout:3779 read 0x26a0a29 thru 0x26ad39f (0xc977 bytes) 2024-05-08T08:25:49.188 INFO:teuthology.orchestra.run.smithi031.stdout:3781 read 0x3d95c31 thru 0x3da5a11 (0xfde1 bytes) 2024-05-08T08:25:49.188 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.188 INFO:teuthology.orchestra.run.smithi031.stdout:3783 punch from 0x284089 to 0x2847f1, (0x768 bytes) 2024-05-08T08:25:49.188 INFO:teuthology.orchestra.run.smithi031.stdout:3785 read 0x8386042 thru 0x838e19a (0x8159 bytes) 2024-05-08T08:25:49.188 INFO:teuthology.orchestra.run.smithi031.stdout:3786 read 0x2a7291e thru 0x2a80f18 (0xe5fb bytes) 2024-05-08T08:25:49.189 INFO:teuthology.orchestra.run.smithi031.stdout:3787 punch from 0x187b1ae to 0x187b9bc, (0x80e bytes) 2024-05-08T08:25:49.189 INFO:teuthology.orchestra.run.smithi031.stdout:3788 write 0x868d822 thru 0x868e2af (0xa8e bytes) 2024-05-08T08:25:49.197 INFO:teuthology.orchestra.run.smithi031.stdout:3789 write 0x91a832b thru 0x91acbea (0x48c0 bytes) 2024-05-08T08:25:49.200 INFO:teuthology.orchestra.run.smithi031.stdout:3791 read 0x7dc70ac thru 0x7dd0b1e (0x9a73 bytes) 2024-05-08T08:25:49.203 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:49.203 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:49.208 INFO:teuthology.orchestra.run.smithi031.stdout:3793 write 0xa435405 thru 0xa438b90 (0x378c bytes) 2024-05-08T08:25:49.211 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:49.219+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:49.220 INFO:teuthology.orchestra.run.smithi031.stdout:3794 write 0xc44b049 thru 0xc453925 (0x88dd bytes) 2024-05-08T08:25:49.222 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:49.223 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:49.224 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.224 INFO:teuthology.orchestra.run.smithi031.stdout:3798 write 0x1641502 thru 0x164c71b (0xb21a bytes) 2024-05-08T08:25:49.230 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:49.230 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:49.232 INFO:teuthology.orchestra.run.smithi031.stdout:3801 read 0x8ea2d45 thru 0x8eaa544 (0x7800 bytes) 2024-05-08T08:25:49.233 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:49.247 INFO:teuthology.orchestra.run.smithi031.stdout:3803 trunc from 0xc453926 to 0x76dac2 2024-05-08T08:25:49.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:49.250 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:49.284 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:49.284 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:49.292 INFO:teuthology.orchestra.run.smithi031.stdout:3804 punch from 0x53c48f to 0x53d5b7, (0x1128 bytes) 2024-05-08T08:25:49.292 INFO:teuthology.orchestra.run.smithi031.stdout:3805 read 0x676fe thru 0x6d980 (0x6283 bytes) 2024-05-08T08:25:49.301 INFO:teuthology.orchestra.run.smithi031.stdout:3806 trunc from 0x76dac2 to 0x6d4d2fc 2024-05-08T08:25:49.307 INFO:teuthology.orchestra.run.smithi031.stdout:3807 read 0x4611505 thru 0x461d3e9 (0xbee5 bytes) 2024-05-08T08:25:49.307 INFO:teuthology.orchestra.run.smithi031.stdout:3808 punch from 0x6486541 to 0x648ab91, (0x4650 bytes) 2024-05-08T08:25:49.307 INFO:teuthology.orchestra.run.smithi031.stdout:3809 punch from 0x4d28146 to 0x4d361f8, (0xe0b2 bytes) 2024-05-08T08:25:49.312 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:49.312 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:49.313 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:49.320+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:49.315 INFO:teuthology.orchestra.run.smithi031.stdout:3811 write 0x90fa6e8 thru 0x9103334 (0x8c4d bytes) 2024-05-08T08:25:49.319 INFO:teuthology.orchestra.run.smithi031.stdout:3812 read 0x116aff4 thru 0x1171d62 (0x6d6f bytes) 2024-05-08T08:25:49.320 INFO:teuthology.orchestra.run.smithi031.stdout:3814 write 0x17cba10 thru 0x17cf734 (0x3d25 bytes) 2024-05-08T08:25:49.323 INFO:teuthology.orchestra.run.smithi031.stdout:3816 read 0x2239f96 thru 0x2248211 (0xe27c bytes) 2024-05-08T08:25:49.324 INFO:teuthology.orchestra.run.smithi031.stdout:3817 punch from 0x69db23c to 0x69ddc6d, (0x2a31 bytes) 2024-05-08T08:25:49.324 INFO:teuthology.orchestra.run.smithi031.stdout:3819 read 0x2abb664 thru 0x2ac7f6b (0xc908 bytes) 2024-05-08T08:25:49.324 INFO:teuthology.orchestra.run.smithi031.stdout:3821 read 0x3b9c0a3 thru 0x3b9e9aa (0x2908 bytes) 2024-05-08T08:25:49.325 INFO:teuthology.orchestra.run.smithi031.stdout:3823 trunc from 0x9103335 to 0x764aee8 2024-05-08T08:25:49.336 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:49.336 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:49.336 INFO:teuthology.orchestra.run.smithi031.stdout:3824 read 0x146312f thru 0x147100f (0xdee1 bytes) 2024-05-08T08:25:49.337 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.337 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.339 INFO:teuthology.orchestra.run.smithi031.stdout:3827 trunc from 0x764aee8 to 0x5fe20f2 2024-05-08T08:25:49.346 INFO:teuthology.orchestra.run.smithi031.stdout:3829 trunc from 0x5fe20f2 to 0x5a96cbd 2024-05-08T08:25:49.353 INFO:teuthology.orchestra.run.smithi031.stdout:3830 read 0x400ee0b thru 0x4011c43 (0x2e39 bytes) 2024-05-08T08:25:49.365 INFO:teuthology.orchestra.run.smithi031.stdout:3835 write 0xba90515 thru 0xba95309 (0x4df5 bytes) 2024-05-08T08:25:49.368 INFO:teuthology.orchestra.run.smithi031.stdout:3836 read 0x318f9b5 thru 0x319be78 (0xc4c4 bytes) 2024-05-08T08:25:49.371 INFO:teuthology.orchestra.run.smithi031.stdout:3837 trunc from 0xba9530a to 0xebe0b2f 2024-05-08T08:25:49.377 INFO:teuthology.orchestra.run.smithi031.stdout:3838 write 0x3ee1117 thru 0x3ee4da0 (0x3c8a bytes) 2024-05-08T08:25:49.380 INFO:teuthology.orchestra.run.smithi031.stdout:3840 punch from 0x9fdb4d8 to 0x9fdc686, (0x11ae bytes) 2024-05-08T08:25:49.380 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.390 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:49.390 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:49.394 INFO:teuthology.orchestra.run.smithi031.stdout:3842 trunc from 0xebe0b2f to 0x23b6cfa 2024-05-08T08:25:49.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.405 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:49.410 INFO:teuthology.orchestra.run.smithi031.stdout:3845 write 0x2cd8146 thru 0x2cdbc16 (0x3ad1 bytes) 2024-05-08T08:25:49.413 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:49.421+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:49.431 INFO:teuthology.orchestra.run.smithi031.stdout:3846 write 0xd4a6498 thru 0xd4a80d9 (0x1c42 bytes) 2024-05-08T08:25:49.435 INFO:teuthology.orchestra.run.smithi031.stdout:3847 read 0x68c5e6e thru 0x68d1df5 (0xbf88 bytes) 2024-05-08T08:25:49.435 INFO:teuthology.orchestra.run.smithi031.stdout:3848 write 0x3824bfb thru 0x383422b (0xf631 bytes) 2024-05-08T08:25:49.439 INFO:teuthology.orchestra.run.smithi031.stdout:3849 clone 24 order 18 su 1024 sc 9 2024-05-08T08:25:49.514 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:49.522+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:49.520 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:49.520 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:49.529 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:49.614 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:49.622+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:49.679 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:49.679 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:49.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:49.715 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:49.723+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:49.817 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:49.824+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:49.837 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:49.837 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:49.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:49.918 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:49.925+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:49.957 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:49.957 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:49.965 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:50.019 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:50.027+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:50.074 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:50.075 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:50.083 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:50.120 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:50.128+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:50.166 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-05-08T08:25:50.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-05-08T08:25:50.199 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:50.200 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:50.211 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:50.226 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone23 from 0xd4a80da (overlap 0x76dac2) to 0x0 2024-05-08T08:25:50.227 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:50.234+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:50.277 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-05-08T08:25:50.322 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:50.329+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:50.332 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:50.332 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:50.341 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:50.422 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:50.430+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:50.436 INFO:teuthology.orchestra.run.smithi031.stdout:3850 read 0x45ddef9 thru 0x45e2e46 (0x4f4e bytes) 2024-05-08T08:25:50.437 INFO:teuthology.orchestra.run.smithi031.stdout:3851 writesame 0x300db6c thru 0x3014aa2 data_size 0x1(0x6f37 bytes) 2024-05-08T08:25:50.477 INFO:teuthology.orchestra.run.smithi031.stdout:3852 punch from 0x9e973eb to 0x9ea18ab, (0xa4c0 bytes) 2024-05-08T08:25:50.488 INFO:teuthology.orchestra.run.smithi031.stdout:3853 trunc from 0xd4a80da to 0x3ca5463 2024-05-08T08:25:50.502 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:50.502 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:50.513 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:50.548 INFO:teuthology.orchestra.run.smithi031.stdout:3854 trunc from 0x3ca5463 to 0xcc67992 2024-05-08T08:25:50.555 INFO:teuthology.orchestra.run.smithi031.stdout:3855 punch from 0xcae1fa5 to 0xcaeee36, (0xce91 bytes) 2024-05-08T08:25:50.556 INFO:teuthology.orchestra.run.smithi031.stdout:3857 punch from 0x8c0013f to 0x8c01f2b, (0x1dec bytes) 2024-05-08T08:25:50.556 INFO:teuthology.orchestra.run.smithi031.stdout:3858 write 0xadf7788 thru 0xae00214 (0x8a8d bytes) 2024-05-08T08:25:50.557 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:50.565+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:50.566 INFO:teuthology.orchestra.run.smithi031.stdout:3859 writesame 0xb164ef thru 0xb1c90f data_size 0x1(0x6421 bytes) 2024-05-08T08:25:50.623 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:50.631+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:50.656 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:50.656 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:50.670 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:3861 read 0x5236c54 thru 0x5245b84 (0xef31 bytes) 2024-05-08T08:25:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:3864 read 0x9945c16 thru 0x99489db (0x2dc6 bytes) 2024-05-08T08:25:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:3865 punch from 0x6e83a0d to 0x6e8fbc4, (0xc1b7 bytes) 2024-05-08T08:25:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:3866 read 0x4d01ea7 thru 0x4d102dd (0xe437 bytes) 2024-05-08T08:25:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:3867 read 0x450d244 thru 0x4517edd (0xac9a bytes) 2024-05-08T08:25:50.700 INFO:teuthology.orchestra.run.smithi031.stdout:truncating to largest ever: 0xeddf218 2024-05-08T08:25:50.702 INFO:teuthology.orchestra.run.smithi031.stdout:3868 trunc from 0xcc67992 to 0xeddf218 2024-05-08T08:25:50.707 INFO:teuthology.orchestra.run.smithi031.stdout:3870 read 0x93d4aa6 thru 0x93d6344 (0x189f bytes) 2024-05-08T08:25:50.707 INFO:teuthology.orchestra.run.smithi031.stdout:3871 write 0x2d2f2dd thru 0x2d37a09 (0x872d bytes) 2024-05-08T08:25:50.747 INFO:teuthology.orchestra.run.smithi031.stdout:3872 read 0x860a570 thru 0x8614531 (0x9fc2 bytes) 2024-05-08T08:25:50.747 INFO:teuthology.orchestra.run.smithi031.stdout:3874 read 0xd02fc02 thru 0xd03582e (0x5c2d bytes) 2024-05-08T08:25:50.747 INFO:teuthology.orchestra.run.smithi031.stdout:3875 write 0x9f85b9e thru 0x9f906bc (0xab1f bytes) 2024-05-08T08:25:50.754 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:50.762+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:50.761 INFO:teuthology.orchestra.run.smithi031.stdout:3876 write 0x925696f thru 0x925fc50 (0x92e2 bytes) 2024-05-08T08:25:50.770 INFO:teuthology.orchestra.run.smithi031.stdout:3877 write 0x46925b thru 0x46f310 (0x60b6 bytes) 2024-05-08T08:25:50.825 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:50.832+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:50.866 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:50.867 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:50.879 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:50.963 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:50.971+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:50.976 INFO:teuthology.orchestra.run.smithi031.stdout:3878 write 0xde89e03 thru 0xde8c263 (0x2461 bytes) 2024-05-08T08:25:50.989 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:50.989 INFO:teuthology.orchestra.run.smithi031.stdout:3885 writesame 0x25fe92e thru 0x26028f2 data_size 0x5(0x3fc5 bytes) 2024-05-08T08:25:51.027 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:51.034+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:51.038 INFO:teuthology.orchestra.run.smithi031.stdout:3887 write 0xd9be5ed thru 0xd9c5e6e (0x7882 bytes) 2024-05-08T08:25:51.049 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:51.049 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:51.051 INFO:teuthology.orchestra.run.smithi031.stdout:3888 write 0x947827c thru 0x9480e49 (0x8bce bytes) 2024-05-08T08:25:51.059 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:51.062 INFO:teuthology.orchestra.run.smithi031.stdout:3891 write 0x1cf5d8c thru 0x1cf79a4 (0x1c19 bytes) 2024-05-08T08:25:51.117 INFO:teuthology.orchestra.run.smithi031.stdout:3892 read 0xd215068 thru 0xd219118 (0x40b1 bytes) 2024-05-08T08:25:51.117 INFO:teuthology.orchestra.run.smithi031.stdout:3893 write 0xe5545b8 thru 0xe55d10a (0x8b53 bytes) 2024-05-08T08:25:51.127 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:51.135+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:51.130 INFO:teuthology.orchestra.run.smithi031.stdout:3894 read 0xb38cfc8 thru 0xb39b53f (0xe578 bytes) 2024-05-08T08:25:51.136 INFO:teuthology.orchestra.run.smithi031.stdout:3895 trunc from 0xeddf218 to 0x9df5705 2024-05-08T08:25:51.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:25:51.164 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-05-08T08:25:51.173 INFO:teuthology.orchestra.run.smithi031.stdout:3898 read 0x20d200b thru 0x20dba0b (0x9a01 bytes) 2024-05-08T08:25:51.181 INFO:teuthology.orchestra.run.smithi031.stdout:3899 writesame 0x69d41c2 thru 0x69da0e5 data_size 0x2(0x5f24 bytes) 2024-05-08T08:25:51.182 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T08:25:51.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:3900 read 0x76059da thru 0x760a88b (0x4eb2 bytes) 2024-05-08T08:25:51.195 INFO:teuthology.orchestra.run.smithi031.stdout:3901 write 0x376eb2c thru 0x3774c6a (0x613f bytes) 2024-05-08T08:25:51.228 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:51.236+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:51.229 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:51.229 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:51.240 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:3902 punch from 0x3b453cc to 0x3b48c29, (0x385d bytes) 2024-05-08T08:25:51.244 INFO:teuthology.orchestra.run.smithi031.stdout:3904 write 0x169ba06 thru 0x16a121a (0x5815 bytes) 2024-05-08T08:25:51.308 INFO:teuthology.orchestra.run.smithi031.stdout:3905 write 0xbba6cad thru 0xbba9c70 (0x2fc4 bytes) 2024-05-08T08:25:51.324 INFO:teuthology.orchestra.run.smithi031.stdout:3906 read 0x2148135 thru 0x214c4a0 (0x436c bytes) 2024-05-08T08:25:51.329 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:51.336+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:51.331 INFO:teuthology.orchestra.run.smithi031.stdout:3907 trunc from 0xbba9c71 to 0x6be335d 2024-05-08T08:25:51.351 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:51.369 INFO:teuthology.orchestra.run.smithi031.stdout:3908 write 0xc92c22f thru 0xc93ad4f (0xeb21 bytes) 2024-05-08T08:25:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:3909 punch from 0x63b8a77 to 0x63c4a59, (0xbfe2 bytes) 2024-05-08T08:25:51.383 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.384 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.384 INFO:teuthology.orchestra.run.smithi031.stdout:3915 write 0x8e7c6ee thru 0x8e8a1ec (0xdaff bytes) 2024-05-08T08:25:51.401 INFO:teuthology.orchestra.run.smithi031.stdout:3917 writesame 0x8c3b8af thru 0x8c3fd6a data_size 0x53(0x44bc bytes) 2024-05-08T08:25:51.403 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:51.404 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:51.412 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.412 INFO:teuthology.orchestra.run.smithi031.stdout:3920 punch from 0x90b5933 to 0x90c4325, (0xe9f2 bytes) 2024-05-08T08:25:51.412 INFO:teuthology.orchestra.run.smithi031.stdout:3921 read 0xa9c45d9 thru 0xa9cdfac (0x99d4 bytes) 2024-05-08T08:25:51.412 INFO:teuthology.orchestra.run.smithi031.stdout:3922 write 0x2a0a5b5 thru 0x2a12bc1 (0x860d bytes) 2024-05-08T08:25:51.415 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:51.429 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:51.437+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:51.454 INFO:teuthology.orchestra.run.smithi031.stdout:3923 punch from 0x6a56edf to 0x6a6575d, (0xe87e bytes) 2024-05-08T08:25:51.457 INFO:teuthology.orchestra.run.smithi031.stdout:3924 trunc from 0xc93ad50 to 0x9cd37c1 2024-05-08T08:25:51.471 INFO:teuthology.orchestra.run.smithi031.stdout:3925 punch from 0x82f6a3 to 0x8355c7, (0x5f24 bytes) 2024-05-08T08:25:51.471 INFO:teuthology.orchestra.run.smithi031.stdout:3926 read 0x6ef2611 thru 0x6ef92fd (0x6ced bytes) 2024-05-08T08:25:51.471 INFO:teuthology.orchestra.run.smithi031.stdout:3927 writesame 0xe23139 thru 0xe2ccaa data_size 0x65(0x9b72 bytes) 2024-05-08T08:25:51.531 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:51.538+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:51.548 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:51.548 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:51.557 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:51.557 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:51.567 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:51.570 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:51.570 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:51.592 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:51.592 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:51.605 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:51.606 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:51.610 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:51.610 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:51.619 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-05-08T08:25:51.172468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":895458971,"num_objects":1124,"num_object_clones":261,"num_object_copies":2248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1124,"num_whiteouts":462,"num_read":60278,"num_read_kb":610207,"num_write":12231,"num_write_kb":353389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9552,"ondisk_log_size":9552,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":358144,"kb_used_data":50120,"kb_used_omap":24,"kb_used_meta":307879,"kb_avail":754616576,"statfs":{"total":773094113280,"available":772727373824,"internally_reserved":0,"allocated":51322880,"data_stored":522550643,"data_compressed":16339758,"data_compressed_allocated":39747584,"data_compressed_original":516371822,"omap_allocated":24704,"internal_metadata":315268992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":74831309,"num_objects":94,"num_object_clones":36,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":11,"num_read":8052,"num_read_kb":202107,"num_write":728,"num_write_kb":57134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023079"},"pg_stats":[{"pgid":"3.1f","version":"60'292","reported_seq":1782,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.267575+0000","last_change":"2024-05-08T08:25:06.628835+0000","last_active":"2024-05-08T08:25:50.267575+0000","last_peered":"2024-05-08T08:25:50.267575+0000","last_clean":"2024-05-08T08:25:50.267575+0000","last_became_active":"2024-05-08T08:25:04.610836+0000","last_became_peered":"2024-05-08T08:25:04.610836+0000","last_unstale":"2024-05-08T08:25:50.267575+0000","last_undegraded":"2024-05-08T08:25:50.267575+0000","last_fullsized":"2024-05-08T08:25:50.267575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:12.849579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011177400000000001,"stat_sum":{"num_bytes":20581408,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":3393,"num_read_kb":15646,"num_write":279,"num_write_kb":6170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'218","reported_seq":1033,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.448840+0000","last_change":"2024-05-08T08:25:06.611278+0000","last_active":"2024-05-08T08:25:50.448840+0000","last_peered":"2024-05-08T08:25:50.448840+0000","last_clean":"2024-05-08T08:25:50.448840+0000","last_became_active":"2024-05-08T08:25:04.611994+0000","last_became_peered":"2024-05-08T08:25:04.611994+0000","last_unstale":"2024-05-08T08:25:50.448840+0000","last_undegraded":"2024-05-08T08:25:50.448840+0000","last_fullsized":"2024-05-08T08:25:50.448840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:31.242286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010168530000000001,"stat_sum":{"num_bytes":19775175,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":997,"num_read_kb":21951,"num_write":262,"num_write_kb":15843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"59'202","reported_seq":1102,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.314972+0000","last_change":"2024-05-08T08:25:06.625063+0000","last_active":"2024-05-08T08:25:50.314972+0000","last_peered":"2024-05-08T08:25:50.314972+0000","last_clean":"2024-05-08T08:25:50.314972+0000","last_became_active":"2024-05-08T08:25:04.610134+0000","last_became_peered":"2024-05-08T08:25:04.610134+0000","last_unstale":"2024-05-08T08:25:50.314972+0000","last_undegraded":"2024-05-08T08:25:50.314972+0000","last_fullsized":"2024-05-08T08:25:50.314972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:11.796747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005171,"stat_sum":{"num_bytes":14449780,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":1261,"num_read_kb":17820,"num_write":216,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"60'207","reported_seq":1855,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.269796+0000","last_change":"2024-05-08T08:25:06.625044+0000","last_active":"2024-05-08T08:25:50.269796+0000","last_peered":"2024-05-08T08:25:50.269796+0000","last_clean":"2024-05-08T08:25:50.269796+0000","last_became_active":"2024-05-08T08:25:04.615120+0000","last_became_peered":"2024-05-08T08:25:04.615120+0000","last_unstale":"2024-05-08T08:25:50.269796+0000","last_undegraded":"2024-05-08T08:25:50.269796+0000","last_fullsized":"2024-05-08T08:25:50.269796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:13:07.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010445210000000001,"stat_sum":{"num_bytes":94835931,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":1976,"num_read_kb":56801,"num_write":265,"num_write_kb":21749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'935","reported_seq":1478,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.172125+0000","last_change":"2024-05-08T08:25:06.628198+0000","last_active":"2024-05-08T08:25:50.172125+0000","last_peered":"2024-05-08T08:25:50.172125+0000","last_clean":"2024-05-08T08:25:50.172125+0000","last_became_active":"2024-05-08T08:25:04.612215+0000","last_became_peered":"2024-05-08T08:25:04.612215+0000","last_unstale":"2024-05-08T08:25:50.172125+0000","last_undegraded":"2024-05-08T08:25:50.172125+0000","last_fullsized":"2024-05-08T08:25:50.172125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:26.847029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064530100000000003,"stat_sum":{"num_bytes":18110614,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":5228,"num_read_kb":5326,"num_write":1545,"num_write_kb":2587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"60'255","reported_seq":2320,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.283008+0000","last_change":"2024-05-08T08:25:48.158059+0000","last_active":"2024-05-08T08:25:50.283008+0000","last_peered":"2024-05-08T08:25:50.283008+0000","last_clean":"2024-05-08T08:25:50.283008+0000","last_became_active":"2024-05-08T08:25:48.157597+0000","last_became_peered":"2024-05-08T08:25:48.157597+0000","last_unstale":"2024-05-08T08:25:50.283008+0000","last_undegraded":"2024-05-08T08:25:50.283008+0000","last_fullsized":"2024-05-08T08:25:50.283008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:27.613891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080728699999999996,"stat_sum":{"num_bytes":45666384,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":2200,"num_read_kb":18882,"num_write":314,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"56'182","reported_seq":1010,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.152417+0000","last_change":"2024-05-08T08:25:47.152061+0000","last_active":"2024-05-08T08:25:47.152061+0000","last_peered":"2024-05-08T08:25:46.720808+0000","last_clean":"2024-05-08T08:25:18.944546+0000","last_became_active":"2024-05-08T08:25:46.258728+0000","last_became_peered":"2024-05-08T08:25:46.258728+0000","last_unstale":"2024-05-08T08:25:47.152417+0000","last_undegraded":"2024-05-08T08:25:47.152417+0000","last_fullsized":"2024-05-08T08:25:47.152417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040135600000000001,"stat_sum":{"num_bytes":17597145,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":1190,"num_read_kb":4735,"num_write":186,"num_write_kb":6959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'219","reported_seq":787,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.313207+0000","last_change":"2024-05-08T08:25:06.610175+0000","last_active":"2024-05-08T08:25:50.313207+0000","last_peered":"2024-05-08T08:25:50.313207+0000","last_clean":"2024-05-08T08:25:50.313207+0000","last_became_active":"2024-05-08T08:25:04.609185+0000","last_became_peered":"2024-05-08T08:25:04.609185+0000","last_unstale":"2024-05-08T08:25:50.313207+0000","last_undegraded":"2024-05-08T08:25:50.313207+0000","last_fullsized":"2024-05-08T08:25:50.313207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:39:14.749019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505687,"stat_sum":{"num_bytes":11945945,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":1053,"num_read_kb":13308,"num_write":232,"num_write_kb":12783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'568","reported_seq":1337,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.247719+0000","last_change":"2024-05-08T08:25:06.628387+0000","last_active":"2024-05-08T08:25:50.247719+0000","last_peered":"2024-05-08T08:25:50.247719+0000","last_clean":"2024-05-08T08:25:50.247719+0000","last_became_active":"2024-05-08T08:25:04.610818+0000","last_became_peered":"2024-05-08T08:25:04.610818+0000","last_unstale":"2024-05-08T08:25:50.247719+0000","last_undegraded":"2024-05-08T08:25:50.247719+0000","last_fullsized":"2024-05-08T08:25:50.247719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:34.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069309899999999997,"stat_sum":{"num_bytes":30029466,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3146,"num_read_kb":6177,"num_write":879,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"60'185","reported_seq":1057,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.312388+0000","last_change":"2024-05-08T08:25:06.625296+0000","last_active":"2024-05-08T08:25:50.312388+0000","last_peered":"2024-05-08T08:25:50.312388+0000","last_clean":"2024-05-08T08:25:50.312388+0000","last_became_active":"2024-05-08T08:25:04.605228+0000","last_became_peered":"2024-05-08T08:25:04.605228+0000","last_unstale":"2024-05-08T08:25:50.312388+0000","last_undegraded":"2024-05-08T08:25:50.312388+0000","last_fullsized":"2024-05-08T08:25:50.312388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:48:10.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011464940000000001,"stat_sum":{"num_bytes":10913207,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":1511,"num_read_kb":12774,"num_write":163,"num_write_kb":7914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"60'239","reported_seq":1582,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239393+0000","last_change":"2024-05-08T08:25:06.796215+0000","last_active":"2024-05-08T08:25:50.239393+0000","last_peered":"2024-05-08T08:25:50.239393+0000","last_clean":"2024-05-08T08:25:50.239393+0000","last_became_active":"2024-05-08T08:25:04.606180+0000","last_became_peered":"2024-05-08T08:25:04.606180+0000","last_unstale":"2024-05-08T08:25:50.239393+0000","last_undegraded":"2024-05-08T08:25:50.239393+0000","last_fullsized":"2024-05-08T08:25:50.239393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:59.196483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027126400000000001,"stat_sum":{"num_bytes":30398012,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":2362,"num_read_kb":16681,"num_write":271,"num_write_kb":4648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"59'333","reported_seq":753,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174111+0000","last_change":"2024-05-08T08:25:06.615415+0000","last_active":"2024-05-08T08:25:50.174111+0000","last_peered":"2024-05-08T08:25:50.174111+0000","last_clean":"2024-05-08T08:25:50.174111+0000","last_became_active":"2024-05-08T08:25:04.614760+0000","last_became_peered":"2024-05-08T08:25:04.614760+0000","last_unstale":"2024-05-08T08:25:50.174111+0000","last_undegraded":"2024-05-08T08:25:50.174111+0000","last_fullsized":"2024-05-08T08:25:50.174111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:51.380106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034964000000000002,"stat_sum":{"num_bytes":15590732,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1270,"num_read_kb":6723,"num_write":478,"num_write_kb":5875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.175991+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:25:50.175991+0000","last_peered":"2024-05-08T08:25:50.175991+0000","last_clean":"2024-05-08T08:25:50.175991+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:25:50.175991+0000","last_undegraded":"2024-05-08T08:25:50.175991+0000","last_fullsized":"2024-05-08T08:25:50.175991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:32.338505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"60'200","reported_seq":697,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.449596+0000","last_change":"2024-05-08T08:25:06.625323+0000","last_active":"2024-05-08T08:25:50.449596+0000","last_peered":"2024-05-08T08:25:50.449596+0000","last_clean":"2024-05-08T08:25:50.449596+0000","last_became_active":"2024-05-08T08:25:04.606855+0000","last_became_peered":"2024-05-08T08:25:04.606855+0000","last_unstale":"2024-05-08T08:25:50.449596+0000","last_undegraded":"2024-05-08T08:25:50.449596+0000","last_fullsized":"2024-05-08T08:25:50.449596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:16.237935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011708910000000001,"stat_sum":{"num_bytes":16728566,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":527,"num_read_kb":6360,"num_write":202,"num_write_kb":8951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.176035+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:25:50.176035+0000","last_peered":"2024-05-08T08:25:50.176035+0000","last_clean":"2024-05-08T08:25:50.176035+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:25:50.176035+0000","last_undegraded":"2024-05-08T08:25:50.176035+0000","last_fullsized":"2024-05-08T08:25:50.176035+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:54.867771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"52'247","reported_seq":1489,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.151756+0000","last_change":"2024-05-08T08:25:47.151756+0000","last_active":"2024-05-08T08:25:45.225592+0000","last_peered":"2024-05-08T08:25:43.120530+0000","last_clean":"2024-05-08T08:25:18.938588+0000","last_became_active":"2024-05-08T08:25:21.002993+0000","last_became_peered":"2024-05-08T08:25:21.002993+0000","last_unstale":"2024-05-08T08:25:47.151756+0000","last_undegraded":"2024-05-08T08:25:47.151756+0000","last_fullsized":"2024-05-08T08:25:47.151756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847699999999999,"stat_sum":{"num_bytes":36763536,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2124,"num_read_kb":25312,"num_write":253,"num_write_kb":4436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174067+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:25:50.174067+0000","last_peered":"2024-05-08T08:25:50.174067+0000","last_clean":"2024-05-08T08:25:50.174067+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:25:50.174067+0000","last_undegraded":"2024-05-08T08:25:50.174067+0000","last_fullsized":"2024-05-08T08:25:50.174067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:43:18.265730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"60'597","reported_seq":2813,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.316168+0000","last_change":"2024-05-08T08:25:06.611208+0000","last_active":"2024-05-08T08:25:50.316168+0000","last_peered":"2024-05-08T08:25:50.316168+0000","last_clean":"2024-05-08T08:25:50.316168+0000","last_became_active":"2024-05-08T08:25:04.608555+0000","last_became_peered":"2024-05-08T08:25:04.608555+0000","last_unstale":"2024-05-08T08:25:50.316168+0000","last_undegraded":"2024-05-08T08:25:50.316168+0000","last_fullsized":"2024-05-08T08:25:50.316168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:04.768564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195331,"stat_sum":{"num_bytes":44612987,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":17,"num_read":5039,"num_read_kb":59082,"num_write":858,"num_write_kb":32480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.206174+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:25:50.206174+0000","last_peered":"2024-05-08T08:25:50.206174+0000","last_clean":"2024-05-08T08:25:50.206174+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:25:50.206174+0000","last_undegraded":"2024-05-08T08:25:50.206174+0000","last_fullsized":"2024-05-08T08:25:50.206174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"59'237","reported_seq":1720,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.318487+0000","last_change":"2024-05-08T08:25:48.159785+0000","last_active":"2024-05-08T08:25:50.318487+0000","last_peered":"2024-05-08T08:25:50.318487+0000","last_clean":"2024-05-08T08:25:50.318487+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:25:50.318487+0000","last_undegraded":"2024-05-08T08:25:50.318487+0000","last_fullsized":"2024-05-08T08:25:50.318487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:07.930167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062277100000000005,"stat_sum":{"num_bytes":35575901,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":2222,"num_read_kb":35423,"num_write":257,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'378","reported_seq":1409,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.171762+0000","last_change":"2024-05-08T08:25:06.628002+0000","last_active":"2024-05-08T08:25:50.171762+0000","last_peered":"2024-05-08T08:25:50.171762+0000","last_clean":"2024-05-08T08:25:50.171762+0000","last_became_active":"2024-05-08T08:25:04.615077+0000","last_became_peered":"2024-05-08T08:25:04.615077+0000","last_unstale":"2024-05-08T08:25:50.171762+0000","last_undegraded":"2024-05-08T08:25:50.171762+0000","last_fullsized":"2024-05-08T08:25:50.171762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.908263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057249000000000004,"stat_sum":{"num_bytes":11737820,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":2433,"num_read_kb":9657,"num_write":442,"num_write_kb":6631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":255,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238459+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:25:50.238459+0000","last_peered":"2024-05-08T08:25:50.238459+0000","last_clean":"2024-05-08T08:25:50.238459+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:25:50.238459+0000","last_undegraded":"2024-05-08T08:25:50.238459+0000","last_fullsized":"2024-05-08T08:25:50.238459+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:25:47.151317+0000","last_change":"2024-05-08T08:25:45.225724+0000","last_active":"2024-05-08T08:25:47.151317+0000","last_peered":"2024-05-08T08:25:47.151317+0000","last_clean":"2024-05-08T08:25:47.151317+0000","last_became_active":"2024-05-08T08:25:45.225388+0000","last_became_peered":"2024-05-08T08:25:45.225388+0000","last_unstale":"2024-05-08T08:25:47.151317+0000","last_undegraded":"2024-05-08T08:25:47.151317+0000","last_fullsized":"2024-05-08T08:25:47.151317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:54.551477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"56'185","reported_seq":883,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.154031+0000","last_change":"2024-05-08T08:25:47.151090+0000","last_active":"2024-05-08T08:25:47.151090+0000","last_peered":"2024-05-08T08:25:46.732061+0000","last_clean":"2024-05-08T08:25:18.936967+0000","last_became_active":"2024-05-08T08:25:46.260405+0000","last_became_peered":"2024-05-08T08:25:46.260405+0000","last_unstale":"2024-05-08T08:25:47.154031+0000","last_undegraded":"2024-05-08T08:25:47.154031+0000","last_fullsized":"2024-05-08T08:25:47.154031+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034192400000000001,"stat_sum":{"num_bytes":23073517,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":580,"num_read_kb":8154,"num_write":223,"num_write_kb":8181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238782+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:25:50.238782+0000","last_peered":"2024-05-08T08:25:50.238782+0000","last_clean":"2024-05-08T08:25:50.238782+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:25:50.238782+0000","last_undegraded":"2024-05-08T08:25:50.238782+0000","last_fullsized":"2024-05-08T08:25:50.238782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"60'210","reported_seq":805,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.314514+0000","last_change":"2024-05-08T08:25:06.615595+0000","last_active":"2024-05-08T08:25:50.314514+0000","last_peered":"2024-05-08T08:25:50.314514+0000","last_clean":"2024-05-08T08:25:50.314514+0000","last_became_active":"2024-05-08T08:25:04.611900+0000","last_became_peered":"2024-05-08T08:25:04.611900+0000","last_unstale":"2024-05-08T08:25:50.314514+0000","last_undegraded":"2024-05-08T08:25:50.314514+0000","last_fullsized":"2024-05-08T08:25:50.314514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:15:47.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050119499999999996,"stat_sum":{"num_bytes":19631352,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":533,"num_read_kb":20098,"num_write":219,"num_write_kb":15522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174391+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:25:50.174391+0000","last_peered":"2024-05-08T08:25:50.174391+0000","last_clean":"2024-05-08T08:25:50.174391+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:25:50.174391+0000","last_undegraded":"2024-05-08T08:25:50.174391+0000","last_fullsized":"2024-05-08T08:25:50.174391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"59'149","reported_seq":1103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.315072+0000","last_change":"2024-05-08T08:25:06.610137+0000","last_active":"2024-05-08T08:25:50.315072+0000","last_peered":"2024-05-08T08:25:50.315072+0000","last_clean":"2024-05-08T08:25:50.315072+0000","last_became_active":"2024-05-08T08:25:04.606801+0000","last_became_peered":"2024-05-08T08:25:04.606801+0000","last_unstale":"2024-05-08T08:25:50.315072+0000","last_undegraded":"2024-05-08T08:25:50.315072+0000","last_fullsized":"2024-05-08T08:25:50.315072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:04.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042592699999999999,"stat_sum":{"num_bytes":61920517,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":899,"num_read_kb":46926,"num_write":151,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238584+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:25:50.238584+0000","last_peered":"2024-05-08T08:25:50.238584+0000","last_clean":"2024-05-08T08:25:50.238584+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:25:50.238584+0000","last_undegraded":"2024-05-08T08:25:50.238584+0000","last_fullsized":"2024-05-08T08:25:50.238584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"56'216","reported_seq":2169,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.206318+0000","last_change":"2024-05-08T08:25:48.159093+0000","last_active":"2024-05-08T08:25:50.206318+0000","last_peered":"2024-05-08T08:25:50.206318+0000","last_clean":"2024-05-08T08:25:50.206318+0000","last_became_active":"2024-05-08T08:25:48.158198+0000","last_became_peered":"2024-05-08T08:25:48.158198+0000","last_unstale":"2024-05-08T08:25:50.206318+0000","last_undegraded":"2024-05-08T08:25:50.206318+0000","last_fullsized":"2024-05-08T08:25:50.206318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.689664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001097338,"stat_sum":{"num_bytes":23653360,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3084,"num_read_kb":21242,"num_write":250,"num_write_kb":16653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"59'218","reported_seq":1078,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174056+0000","last_change":"2024-05-08T08:25:06.615379+0000","last_active":"2024-05-08T08:25:50.174056+0000","last_peered":"2024-05-08T08:25:50.174056+0000","last_clean":"2024-05-08T08:25:50.174056+0000","last_became_active":"2024-05-08T08:25:04.901435+0000","last_became_peered":"2024-05-08T08:25:04.901435+0000","last_unstale":"2024-05-08T08:25:50.174056+0000","last_undegraded":"2024-05-08T08:25:50.174056+0000","last_fullsized":"2024-05-08T08:25:50.174056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:02.472778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039550799999999998,"stat_sum":{"num_bytes":11085840,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2089,"num_read_kb":6401,"num_write":209,"num_write_kb":3268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"60'366","reported_seq":824,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.313079+0000","last_change":"2024-05-08T08:25:06.612336+0000","last_active":"2024-05-08T08:25:50.313079+0000","last_peered":"2024-05-08T08:25:50.313079+0000","last_clean":"2024-05-08T08:25:50.313079+0000","last_became_active":"2024-05-08T08:25:04.615227+0000","last_became_peered":"2024-05-08T08:25:04.615227+0000","last_unstale":"2024-05-08T08:25:50.313079+0000","last_undegraded":"2024-05-08T08:25:50.313079+0000","last_fullsized":"2024-05-08T08:25:50.313079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:23.233411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034968800000000002,"stat_sum":{"num_bytes":28282536,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":1467,"num_read_kb":16390,"num_write":505,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"58'176","reported_seq":1328,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.312050+0000","last_change":"2024-05-08T08:25:06.610868+0000","last_active":"2024-05-08T08:25:50.312050+0000","last_peered":"2024-05-08T08:25:50.312050+0000","last_clean":"2024-05-08T08:25:50.312050+0000","last_became_active":"2024-05-08T08:25:04.608456+0000","last_became_peered":"2024-05-08T08:25:04.608456+0000","last_unstale":"2024-05-08T08:25:50.312050+0000","last_undegraded":"2024-05-08T08:25:50.312050+0000","last_fullsized":"2024-05-08T08:25:50.312050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:37.416621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112149,"stat_sum":{"num_bytes":34120901,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":1096,"num_read_kb":14290,"num_write":176,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"59'192","reported_seq":864,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174647+0000","last_change":"2024-05-08T08:25:06.624428+0000","last_active":"2024-05-08T08:25:50.174647+0000","last_peered":"2024-05-08T08:25:50.174647+0000","last_clean":"2024-05-08T08:25:50.174647+0000","last_became_active":"2024-05-08T08:25:04.606925+0000","last_became_peered":"2024-05-08T08:25:04.606925+0000","last_unstale":"2024-05-08T08:25:50.174647+0000","last_undegraded":"2024-05-08T08:25:50.174647+0000","last_fullsized":"2024-05-08T08:25:50.174647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:24.692160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046376499999999999,"stat_sum":{"num_bytes":58995571,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":614,"num_read_kb":34697,"num_write":264,"num_write_kb":22805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'177","reported_seq":468,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.315221+0000","last_change":"2024-05-08T08:25:06.796419+0000","last_active":"2024-05-08T08:25:50.315221+0000","last_peered":"2024-05-08T08:25:50.315221+0000","last_clean":"2024-05-08T08:25:50.315221+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:25:50.315221+0000","last_undegraded":"2024-05-08T08:25:50.315221+0000","last_fullsized":"2024-05-08T08:25:50.315221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:45.240770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046517299999999999,"stat_sum":{"num_bytes":10995804,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":216,"num_read_kb":7926,"num_write":178,"num_write_kb":4861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"59'322","reported_seq":2341,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238601+0000","last_change":"2024-05-08T08:25:06.796635+0000","last_active":"2024-05-08T08:25:50.238601+0000","last_peered":"2024-05-08T08:25:50.238601+0000","last_clean":"2024-05-08T08:25:50.238601+0000","last_became_active":"2024-05-08T08:25:04.615004+0000","last_became_peered":"2024-05-08T08:25:04.615004+0000","last_unstale":"2024-05-08T08:25:50.238601+0000","last_undegraded":"2024-05-08T08:25:50.238601+0000","last_fullsized":"2024-05-08T08:25:50.238601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:04.429594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060116800000000001,"stat_sum":{"num_bytes":43995266,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":3394,"num_read_kb":37060,"num_write":333,"num_write_kb":22695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"59'213","reported_seq":728,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239137+0000","last_change":"2024-05-08T08:25:06.796737+0000","last_active":"2024-05-08T08:25:50.239137+0000","last_peered":"2024-05-08T08:25:50.239137+0000","last_clean":"2024-05-08T08:25:50.239137+0000","last_became_active":"2024-05-08T08:25:04.615942+0000","last_became_peered":"2024-05-08T08:25:04.615942+0000","last_unstale":"2024-05-08T08:25:50.239137+0000","last_undegraded":"2024-05-08T08:25:50.239137+0000","last_fullsized":"2024-05-08T08:25:50.239137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:54.886102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065917600000000003,"stat_sum":{"num_bytes":5724433,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":794,"num_read_kb":2629,"num_write":252,"num_write_kb":1884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"60'546","reported_seq":1135,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.278796+0000","last_change":"2024-05-08T08:25:06.612297+0000","last_active":"2024-05-08T08:25:50.278796+0000","last_peered":"2024-05-08T08:25:50.278796+0000","last_clean":"2024-05-08T08:25:50.278796+0000","last_became_active":"2024-05-08T08:25:04.616212+0000","last_became_peered":"2024-05-08T08:25:04.616212+0000","last_unstale":"2024-05-08T08:25:50.278796+0000","last_undegraded":"2024-05-08T08:25:50.278796+0000","last_fullsized":"2024-05-08T08:25:50.278796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:31.116080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031704699999999999,"stat_sum":{"num_bytes":5284092,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":2537,"num_read_kb":6619,"num_write":839,"num_write_kb":3465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"59'204","reported_seq":933,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239287+0000","last_change":"2024-05-08T08:25:06.796231+0000","last_active":"2024-05-08T08:25:50.239287+0000","last_peered":"2024-05-08T08:25:50.239287+0000","last_clean":"2024-05-08T08:25:50.239287+0000","last_became_active":"2024-05-08T08:25:04.615031+0000","last_became_peered":"2024-05-08T08:25:04.615031+0000","last_unstale":"2024-05-08T08:25:50.239287+0000","last_undegraded":"2024-05-08T08:25:50.239287+0000","last_fullsized":"2024-05-08T08:25:50.239287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:30:30.412714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040064000000000001,"stat_sum":{"num_bytes":23621913,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":829,"num_read_kb":19159,"num_write":222,"num_write_kb":5194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"58'253","reported_seq":1153,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.316670+0000","last_change":"2024-05-08T08:25:06.628036+0000","last_active":"2024-05-08T08:25:50.316670+0000","last_peered":"2024-05-08T08:25:50.316670+0000","last_clean":"2024-05-08T08:25:50.316670+0000","last_became_active":"2024-05-08T08:25:04.609651+0000","last_became_peered":"2024-05-08T08:25:04.609651+0000","last_unstale":"2024-05-08T08:25:50.316670+0000","last_undegraded":"2024-05-08T08:25:50.316670+0000","last_fullsized":"2024-05-08T08:25:50.316670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:30.467433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050895200000000002,"stat_sum":{"num_bytes":20883517,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":1350,"num_read_kb":20411,"num_write":319,"num_write_kb":13515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"60'514","reported_seq":1361,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.240943+0000","last_change":"2024-05-08T08:25:06.796497+0000","last_active":"2024-05-08T08:25:50.240943+0000","last_peered":"2024-05-08T08:25:50.240943+0000","last_clean":"2024-05-08T08:25:50.240943+0000","last_became_active":"2024-05-08T08:25:04.616040+0000","last_became_peered":"2024-05-08T08:25:04.616040+0000","last_unstale":"2024-05-08T08:25:50.240943+0000","last_undegraded":"2024-05-08T08:25:50.240943+0000","last_fullsized":"2024-05-08T08:25:50.240943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:51.767601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050921500000000004,"stat_sum":{"num_bytes":48287375,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":2748,"num_read_kb":15447,"num_write":830,"num_write_kb":12206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":894868603,"num_objects":1122,"num_object_clones":261,"num_object_copies":2244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1122,"num_whiteouts":462,"num_read":60162,"num_read_kb":610107,"num_write":12072,"num_write_kb":350973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45748224,"data_stored":518901662,"data_compressed":16329872,"data_compressed_allocated":39673856,"data_compressed_original":515224942,"omap_allocated":9107,"internal_metadata":0},"log_size":9435,"ondisk_log_size":9435,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47032,"kb_used_data":7528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324808,"statfs":{"total":96636764160,"available":96588603392,"internally_reserved":0,"allocated":7708672,"data_stored":85619922,"data_compressed":2756454,"data_compressed_allocated":6094848,"data_compressed_original":84713283,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":2551808,"data_stored":21471733,"data_compressed":615042,"data_compressed_allocated":1548288,"data_compressed_original":21004288,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43352,"kb_used_data":8016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328488,"statfs":{"total":96636764160,"available":96592371712,"internally_reserved":0,"allocated":8208384,"data_stored":85986856,"data_compressed":2559614,"data_compressed_allocated":6623232,"data_compressed_original":85109415,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53140,"kb_used_data":8072,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318700,"statfs":{"total":96636764160,"available":96582348800,"internally_reserved":0,"allocated":8265728,"data_stored":85814967,"data_compressed":2840977,"data_compressed_allocated":6426624,"data_compressed_original":84793846,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45168,"kb_used_data":9816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326672,"statfs":{"total":96636764160,"available":96590512128,"internally_reserved":0,"allocated":10051584,"data_stored":103736403,"data_compressed":3214260,"data_compressed_allocated":8335360,"data_compressed_original":102821368,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33500,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338340,"statfs":{"total":96636764160,"available":96602460160,"internally_reserved":0,"allocated":2375680,"data_stored":16593268,"data_compressed":721391,"data_compressed_allocated":1368064,"data_compressed_original":16076703,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":54,"seq":231928233986,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33676,"kb_used_data":4992,"kb_used_omap":13,"kb_used_meta":28658,"kb_avail":94338164,"statfs":{"total":96636764160,"available":96602279936,"internally_reserved":0,"allocated":5111808,"data_stored":46096708,"data_compressed":1232659,"data_compressed_allocated":3903488,"data_compressed_original":45473792,"omap_allocated":13550,"internal_metadata":29346578},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68588,"kb_used_data":6884,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94303252,"statfs":{"total":96636764160,"available":96566530048,"internally_reserved":0,"allocated":7049216,"data_stored":77230786,"data_compressed":2399361,"data_compressed_allocated":5447680,"data_compressed_original":76379127,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6369280,"data_stored":76922333,"data_compressed":2399361,"data_compressed_allocated":5447680,"data_compressed_original":76379127,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4395008,"data_stored":45209515,"data_compressed":1227716,"data_compressed_allocated":3866624,"data_compressed_original":44900352,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1679360,"data_stored":16278634,"data_compressed":721391,"data_compressed_allocated":1368064,"data_compressed_original":16076703,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9375744,"data_stored":103428184,"data_compressed":3214260,"data_compressed_allocated":8335360,"data_compressed_original":102821368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7581696,"data_stored":85506484,"data_compressed":2840977,"data_compressed_allocated":6426624,"data_compressed_original":84793846,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7524352,"data_stored":85678342,"data_compressed":2559614,"data_compressed_allocated":6623232,"data_compressed_original":85109415,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":21157099,"data_compressed":615042,"data_compressed_allocated":1548288,"data_compressed_original":21004288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6967296,"data_stored":84721071,"data_compressed":2751511,"data_compressed_allocated":6057984,"data_compressed_original":84139843,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:51.621 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:51.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.625 INFO:teuthology.orchestra.run.smithi031.stdout:3929 read 0x7667992 thru 0x7672155 (0xa7c4 bytes) 2024-05-08T08:25:51.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.625 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.625 INFO:teuthology.orchestra.run.smithi031.stdout:3936 write 0x261781f thru 0x2620011 (0x87f3 bytes) 2024-05-08T08:25:51.628 INFO:teuthology.orchestra.run.smithi031.stdout:3937 write 0x3782fd2 thru 0x3788bb6 (0x5be5 bytes) 2024-05-08T08:25:51.631 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:51.639+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:51.632 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:51.632 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:51.639 INFO:teuthology.orchestra.run.smithi031.stdout:3938 write 0xa437385 thru 0xa444b84 (0xd800 bytes) 2024-05-08T08:25:51.660 INFO:teuthology.orchestra.run.smithi031.stdout:3940 punch from 0x6cfa346 to 0x6d01ec0, (0x7b7a bytes) 2024-05-08T08:25:51.660 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:51.661 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:51.665 INFO:teuthology.orchestra.run.smithi031.stdout:3942 writesame 0xa50abf7 thru 0xa519804 data_size 0x5(0xec0e bytes) 2024-05-08T08:25:51.668 INFO:teuthology.orchestra.run.smithi031.stdout:3943 write 0x89f1d1e thru 0x89f69db (0x4cbe bytes) 2024-05-08T08:25:51.681 INFO:teuthology.orchestra.run.smithi031.stdout:3944 read 0x7a65cb4 thru 0x7a677a3 (0x1af0 bytes) 2024-05-08T08:25:51.681 INFO:teuthology.orchestra.run.smithi031.stdout:3945 writesame 0x87cb48a thru 0x87d1cd7 data_size 0x4f(0x684e bytes) 2024-05-08T08:25:51.684 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:51.684 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:3947 read 0x5552deb thru 0x55625f5 (0xf80b bytes) 2024-05-08T08:25:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:3949 read 0x9e5ad4a thru 0x9e677b1 (0xca68 bytes) 2024-05-08T08:25:51.695 INFO:teuthology.orchestra.run.smithi031.stdout:3952 writesame 0x2c4f68f thru 0x2c58650 data_size 0x1(0x8fc2 bytes) 2024-05-08T08:25:51.697 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1815 3.10 3:09da06bb:::rbd_data.119116cc7dd0.0000000000000146:head [set-alloc-hint object_size 262144 write_size 262144,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 41.462540619000002, 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057340569999999999, 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.698 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1815 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241200+0000", 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241202+0000", 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9290000000000001e-06 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241211+0000", 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6940000000000001e-06 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241228+0000", 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7487e-05 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241239+0000", 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0954999999999999e-05 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241305+0000", 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5696000000000003e-05 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.241432+0000", 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000127639 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:51.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.242178+0000", 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00074582499999999998 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.245608+0000", 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003430191 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246869+0000", 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001261096 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246904+0000", 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5024999999999998e-05 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.246934+0000", 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9519999999999999e-05 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:1992 3.a 3:525853b5:::rbd_data.119116cc7dd0.0000000000000148:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 41.288801991, 2024-05-08T08:25:51.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005824053, 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1992 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414939+0000", 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414940+0000", 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.029e-06 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414946+0000", 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2890000000000003e-06 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414957+0000", 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1280999999999999e-05 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.414966+0000", 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415026+0000", 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0624000000000001e-05 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415093+0000", 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6752000000000001e-05 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.415154+0000", 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1016e-05 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.417565+0000", 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024105730000000001 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420666+0000", 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031012219999999998 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420721+0000", 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5260000000000003e-05 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:10.420763+0000", 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1519e-05 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.708 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7101 3.a 3:570608b4:::rbd_data.119120fdbc10.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1310720~19922944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 35.488671525999997, 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0053734150000000003, 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7101 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:51.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215069+0000", 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.215071+0000", 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4210000000000002e-06 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217644+0000", 2024-05-08T08:25:51.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0025721239999999999 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217664+0000", 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0492999999999998e-05 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217681+0000", 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6985000000000001e-05 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217759+0000", 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7749000000000004e-05 2024-05-08T08:25:51.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.217900+0000", 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000141361 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.218122+0000", 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022196499999999999 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220140+0000", 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020175129999999999 2024-05-08T08:25:51.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220373+0000", 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023305999999999999 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220410+0000", 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7146e-05 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.220442+0000", 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2598000000000002e-05 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24991 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.257982274, 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005717364, 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.714 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24991 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445758+0000", 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445759+0000", 2024-05-08T08:25:51.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445760+0000", 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.796e-06 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445764+0000", 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7579999999999998e-06 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445767+0000", 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8380000000000002e-06 2024-05-08T08:25:51.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448156+0000", 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0023889200000000001 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448220+0000", 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4469999999999998e-05 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.448567+0000", 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00034631100000000003 2024-05-08T08:25:51.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450776+0000", 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022094570000000002 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451411+0000", 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00063536400000000002 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451442+0000", 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0080999999999999e-05 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451476+0000", 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4088999999999998e-05 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24992 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.25796313, 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0058446770000000004, 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.719 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24992 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445777+0000", 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445778+0000", 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445780+0000", 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.367e-06 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445785+0000", 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4830000000000001e-06 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445788+0000", 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0280000000000001e-06 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449082+0000", 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0032946540000000002 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449148+0000", 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5945999999999995e-05 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.449434+0000", 2024-05-08T08:25:51.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00028535499999999999 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451491+0000", 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020569260000000002 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451531+0000", 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0187e-05 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451583+0000", 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2420999999999998e-05 2024-05-08T08:25:51.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451622+0000", 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8930999999999997e-05 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:24993 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.257934744, 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.006203943, 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.724 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 24993 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:51.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445806+0000", 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445809+0000", 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5000000000000002e-06 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445814+0000", 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3680000000000001e-06 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.445818+0000", 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6169999999999998e-06 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450046+0000", 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042282190000000001 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450131+0000", 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4747999999999996e-05 2024-05-08T08:25:51.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.450449+0000", 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000318022 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451647+0000", 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0011983479999999999 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451925+0000", 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00027754599999999999 2024-05-08T08:25:51.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.451977+0000", 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2355000000000003e-05 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.452010+0000", 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2804e-05 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.729 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25148 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.045208513999999, 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073557630000000004, 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25148 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658532+0000", 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.03e-07 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658535+0000", 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1919999999999999e-06 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658539+0000", 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6779999999999999e-06 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658542+0000", 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.225e-06 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.659927+0000", 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013846729999999999 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660009+0000", 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1887999999999999e-05 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660392+0000", 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00038333600000000002 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665776+0000", 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.005383423 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665836+0000", 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0297000000000002e-05 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665863+0000", 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7412000000000001e-05 2024-05-08T08:25:51.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665888+0000", 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4335999999999999e-05 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25149 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.045188467999999, 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073836789999999998, 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.735 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25149 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:51.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658552+0000", 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7e-07 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658554+0000", 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.877e-06 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658559+0000", 2024-05-08T08:25:51.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5410000000000002e-06 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658562+0000", 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8509999999999999e-06 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660808+0000", 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0022458589999999998 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.660858+0000", 2024-05-08T08:25:51.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0250000000000002e-05 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661053+0000", 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000195269 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665807+0000", 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0047536710000000001 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665894+0000", 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7678999999999999e-05 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665917+0000", 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2921999999999998e-05 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665936+0000", 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8389999999999998e-05 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25150 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.045170159000001, 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074116290000000003, 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25150 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658570+0000", 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658571+0000", 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7099999999999998e-07 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658574+0000", 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9069999999999999e-06 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658580+0000", 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5749999999999997e-06 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658585+0000", 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999998e-06 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661498+0000", 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0029126820000000002 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661544+0000", 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6585999999999997e-05 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.661742+0000", 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019782600000000001 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665830+0000", 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0040875310000000002 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665941+0000", 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110975 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665963+0000", 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.213e-05 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665982+0000", 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9069e-05 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25151 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 17.045150907, 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074729280000000002, 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25151 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658590+0000", 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.35e-07 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658592+0000", 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.855e-06 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658595+0000", 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7570000000000002e-06 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.658598+0000", 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5900000000000002e-06 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662062+0000", 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0034643909999999998 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662148+0000", 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5148999999999994e-05 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.662351+0000", 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000203313 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.665987+0000", 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0036361060000000001 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666009+0000", 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2098000000000001e-05 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666040+0000", 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0660999999999997e-05 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.666063+0000", 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2773e-05 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25290 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.751 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.904228410999998, 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0059807180000000003, 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25290 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799512+0000", 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799513+0000", 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3300000000000001e-07 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799518+0000", 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3090000000000003e-06 2024-05-08T08:25:51.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799560+0000", 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1578000000000001e-05 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799568+0000", 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2849999999999995e-06 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800054+0000", 2024-05-08T08:25:51.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000486121 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800137+0000", 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3003000000000005e-05 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800339+0000", 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00020165600000000001 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803776+0000", 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.003436691 2024-05-08T08:25:51.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805429+0000", 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0016535779999999999 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805461+0000", 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1674999999999997e-05 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805493+0000", 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1989000000000002e-05 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.756 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25291 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.903790335, 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0056878800000000002, 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25291 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.757 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799950+0000", 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799951+0000", 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9100000000000003e-07 2024-05-08T08:25:51.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799955+0000", 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2379999999999999e-06 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799964+0000", 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3680000000000008e-06 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.799970+0000", 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9939999999999997e-06 2024-05-08T08:25:51.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800828+0000", 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00085768499999999996 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800877+0000", 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8609000000000002e-05 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801075+0000", 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019808400000000001 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805212+0000", 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0041372850000000001 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805543+0000", 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00033049199999999999 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805595+0000", 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1919999999999998e-05 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.805638+0000", 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3513999999999997e-05 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25293 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.902829582999999, 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0073904310000000003, 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.762 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25293 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800911+0000", 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800912+0000", 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800916+0000", 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9689999999999996e-06 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800924+0000", 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3140000000000004e-06 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800930+0000", 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3450000000000002e-06 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801990+0000", 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00106022 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802073+0000", 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2761000000000003e-05 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802285+0000", 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000212036 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808000+0000", 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0057151379999999998 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808232+0000", 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023199399999999999 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808264+0000", 2024-05-08T08:25:51.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1851999999999999e-05 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808301+0000", 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7122000000000001e-05 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25294 3.a 3:536ed5cb:::rbd_data.11911e381319.000000000000000e:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.902795574999999, 2024-05-08T08:25:51.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0074925599999999997, 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25294 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800945+0000", 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.77e-07 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800948+0000", 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3750000000000001e-06 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800955+0000", 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2660000000000004e-06 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.800958+0000", 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1599999999999998e-06 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802642+0000", 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001683534 2024-05-08T08:25:51.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802747+0000", 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010508799999999999 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.802853+0000", 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000106428 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808022+0000", 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0051684570000000004 2024-05-08T08:25:51.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808352+0000", 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00032994200000000001 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808398+0000", 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-05 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808438+0000", 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9223e-05 2024-05-08T08:25:51.772 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25295 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.902444681999999, 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0072563819999999996, 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25295 2024-05-08T08:25:51.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801296+0000", 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801297+0000", 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5300000000000003e-07 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801302+0000", 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1399999999999999e-06 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801314+0000", 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2652e-05 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.801321+0000", 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0899999999999999e-06 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803076+0000", 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001754802 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803156+0000", 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9699999999999999e-05 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.803369+0000", 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021332400000000001 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808025+0000", 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0046552039999999996 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808480+0000", 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00045548700000000003 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808515+0000", 2024-05-08T08:25:51.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4782e-05 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.808552+0000", 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7447999999999998e-05 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:25323 3.a 3:56571840:::rbd_object_map.11911e381319:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 16.864508247, 2024-05-08T08:25:51.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0055662289999999998, 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 25323 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839232+0000", 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839233+0000", 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839235+0000", 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2460000000000002e-06 2024-05-08T08:25:51.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839239+0000", 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839242+0000", 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9100000000000001e-06 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839832+0000", 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00058972500000000004 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.839911+0000", 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9762e-05 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.840110+0000", 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019887299999999999 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.841657+0000", 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001547011 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844711+0000", 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0030535689999999999 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844763+0000", 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1952e-05 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:34.844799+0000", 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5627000000000003e-05 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.783 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:27262 3.10 3:0c3194f4:::rbd_data.119137f21733.000000000000020b:head [delete] snapc 13=[13] ondisk+write+known_if_redirected+supports_pool_eio e48)", 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.503170667999999, 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054741219999999997, 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.784 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 27262 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200570+0000", 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200571+0000", 2024-05-08T08:25:51.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4389999999999999e-06 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200576+0000", 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6e-06 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200587+0000", 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1209000000000001e-05 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200592+0000", 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3190000000000002e-06 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200642+0000", 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9870999999999997e-05 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200692+0000", 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9651999999999998e-05 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.200787+0000", 2024-05-08T08:25:51.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5424e-05 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.202753+0000", 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0019658499999999999 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.205947+0000", 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031941119999999998 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.206001+0000", 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3573999999999999e-05 2024-05-08T08:25:51.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:38.206044+0000", 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3071999999999997e-05 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:33615 3.a 3:51fec57b:::rbd_data.11915c46bf0c.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:48.268696+0000", 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.4350441580000002, 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088264999999999993, 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.789 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 33615 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.268696+0000", 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.268696+0000", 2024-05-08T08:25:51.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.268698+0000", 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.708e-06 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271189+0000", 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0024906249999999998 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271199+0000", 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0475999999999999e-05 2024-05-08T08:25:51.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271204+0000", 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1250000000000001e-06 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271259+0000", 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5130000000000002e-05 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271399+0000", 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000139827 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271662+0000", 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00026261200000000001 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277118+0000", 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0054559409999999997 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277406+0000", 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00028822 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277482+0000", 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5694999999999999e-05 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277523+0000", 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1140999999999998e-05 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:33616 3.a 3:51fec57b:::rbd_data.11915c46bf0c.0000000000000001:head [write 0~0] snapc 19=[19] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:51.794 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:48.271222+0000", 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.4325186429999999, 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0066217819999999997, 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 33616 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271222+0000", 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271222+0000", 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271223+0000", 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271225+0000", 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9589999999999998e-06 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271229+0000", 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8600000000000001e-06 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.271233+0000", 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1379999999999999e-06 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.274822+0000", 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0035893499999999998 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.274884+0000", 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1948999999999999e-05 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.275000+0000", 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116223 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277536+0000", 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.002536108 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277760+0000", 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00022422499999999999 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277808+0000", 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7958999999999999e-05 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.277844+0000", 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5283000000000001e-05 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.799 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:35110 3.a 3:57803534:::rbd_data.11915c46bf0c.0000000000000019:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 2097152 write_size 2097152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58)", 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:48.378638+0000", 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 3.325102958, 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0093983020000000007, 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 35110 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.800 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.378638+0000", 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.378638+0000", 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.378641+0000", 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8829999999999999e-06 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.383107+0000", 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0044659549999999998 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.383119+0000", 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1374e-05 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.383126+0000", 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.267e-06 2024-05-08T08:25:51.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.383180+0000", 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4441000000000003e-05 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.383319+0000", 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000138567 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.383648+0000", 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000328444 2024-05-08T08:25:51.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.387936+0000", 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042889699999999996 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.387967+0000", 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0663000000000001e-05 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.387999+0000", 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1940999999999998e-05 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:51.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.388036+0000", 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6797e-05 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:51.805 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:51.806 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:51.807 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:51.807 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:51.814+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:51.808 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:51.808 INFO:teuthology.orchestra.run.smithi031.stdout:3953 write 0x6827386 thru 0x682e3e3 (0x705e bytes) 2024-05-08T08:25:51.808 INFO:teuthology.orchestra.run.smithi031.stdout:3954 trunc from 0xa519805 to 0x8f34889 2024-05-08T08:25:51.808 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.809 INFO:teuthology.orchestra.run.smithi031.stdout:3956 read 0x35ca2aa thru 0x35ce125 (0x3e7c bytes) 2024-05-08T08:25:51.809 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:51.809 INFO:teuthology.orchestra.run.smithi031.stdout:3959 trunc from 0x8f34889 to 0x1844f65 2024-05-08T08:25:51.857 INFO:teuthology.orchestra.run.smithi031.stdout:3960 write 0xd9282da thru 0xd92ded1 (0x5bf8 bytes) 2024-05-08T08:25:51.868 INFO:teuthology.orchestra.run.smithi031.stdout:3961 flatten 2024-05-08T08:25:51.907 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:51.914+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:52.001 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:52.001 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:52.001 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:52.001 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:52.008 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:52.016+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:52.013 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:52.052 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:52.053 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:52.068 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2024-05-08T08:25:51.172468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":895458971,"num_objects":1124,"num_object_clones":261,"num_object_copies":2248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1124,"num_whiteouts":462,"num_read":60278,"num_read_kb":610207,"num_write":12231,"num_write_kb":353389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9552,"ondisk_log_size":9552,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":358144,"kb_used_data":50120,"kb_used_omap":24,"kb_used_meta":307879,"kb_avail":754616576,"statfs":{"total":773094113280,"available":772727373824,"internally_reserved":0,"allocated":51322880,"data_stored":522550643,"data_compressed":16339758,"data_compressed_allocated":39747584,"data_compressed_original":516371822,"omap_allocated":24704,"internal_metadata":315268992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":74831309,"num_objects":94,"num_object_clones":36,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":11,"num_read":8052,"num_read_kb":202107,"num_write":728,"num_write_kb":57134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023079"},"pg_stats":[{"pgid":"3.1f","version":"60'292","reported_seq":1782,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.267575+0000","last_change":"2024-05-08T08:25:06.628835+0000","last_active":"2024-05-08T08:25:50.267575+0000","last_peered":"2024-05-08T08:25:50.267575+0000","last_clean":"2024-05-08T08:25:50.267575+0000","last_became_active":"2024-05-08T08:25:04.610836+0000","last_became_peered":"2024-05-08T08:25:04.610836+0000","last_unstale":"2024-05-08T08:25:50.267575+0000","last_undegraded":"2024-05-08T08:25:50.267575+0000","last_fullsized":"2024-05-08T08:25:50.267575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:12.849579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011177400000000001,"stat_sum":{"num_bytes":20581408,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":3393,"num_read_kb":15646,"num_write":279,"num_write_kb":6170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'218","reported_seq":1033,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.448840+0000","last_change":"2024-05-08T08:25:06.611278+0000","last_active":"2024-05-08T08:25:50.448840+0000","last_peered":"2024-05-08T08:25:50.448840+0000","last_clean":"2024-05-08T08:25:50.448840+0000","last_became_active":"2024-05-08T08:25:04.611994+0000","last_became_peered":"2024-05-08T08:25:04.611994+0000","last_unstale":"2024-05-08T08:25:50.448840+0000","last_undegraded":"2024-05-08T08:25:50.448840+0000","last_fullsized":"2024-05-08T08:25:50.448840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:31.242286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010168530000000001,"stat_sum":{"num_bytes":19775175,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":997,"num_read_kb":21951,"num_write":262,"num_write_kb":15843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"59'202","reported_seq":1102,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.314972+0000","last_change":"2024-05-08T08:25:06.625063+0000","last_active":"2024-05-08T08:25:50.314972+0000","last_peered":"2024-05-08T08:25:50.314972+0000","last_clean":"2024-05-08T08:25:50.314972+0000","last_became_active":"2024-05-08T08:25:04.610134+0000","last_became_peered":"2024-05-08T08:25:04.610134+0000","last_unstale":"2024-05-08T08:25:50.314972+0000","last_undegraded":"2024-05-08T08:25:50.314972+0000","last_fullsized":"2024-05-08T08:25:50.314972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:11.796747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005171,"stat_sum":{"num_bytes":14449780,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":1261,"num_read_kb":17820,"num_write":216,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"60'207","reported_seq":1855,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.269796+0000","last_change":"2024-05-08T08:25:06.625044+0000","last_active":"2024-05-08T08:25:50.269796+0000","last_peered":"2024-05-08T08:25:50.269796+0000","last_clean":"2024-05-08T08:25:50.269796+0000","last_became_active":"2024-05-08T08:25:04.615120+0000","last_became_peered":"2024-05-08T08:25:04.615120+0000","last_unstale":"2024-05-08T08:25:50.269796+0000","last_undegraded":"2024-05-08T08:25:50.269796+0000","last_fullsized":"2024-05-08T08:25:50.269796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:13:07.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010445210000000001,"stat_sum":{"num_bytes":94835931,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":1976,"num_read_kb":56801,"num_write":265,"num_write_kb":21749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'935","reported_seq":1478,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.172125+0000","last_change":"2024-05-08T08:25:06.628198+0000","last_active":"2024-05-08T08:25:50.172125+0000","last_peered":"2024-05-08T08:25:50.172125+0000","last_clean":"2024-05-08T08:25:50.172125+0000","last_became_active":"2024-05-08T08:25:04.612215+0000","last_became_peered":"2024-05-08T08:25:04.612215+0000","last_unstale":"2024-05-08T08:25:50.172125+0000","last_undegraded":"2024-05-08T08:25:50.172125+0000","last_fullsized":"2024-05-08T08:25:50.172125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:26.847029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064530100000000003,"stat_sum":{"num_bytes":18110614,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":5228,"num_read_kb":5326,"num_write":1545,"num_write_kb":2587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"60'255","reported_seq":2320,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.283008+0000","last_change":"2024-05-08T08:25:48.158059+0000","last_active":"2024-05-08T08:25:50.283008+0000","last_peered":"2024-05-08T08:25:50.283008+0000","last_clean":"2024-05-08T08:25:50.283008+0000","last_became_active":"2024-05-08T08:25:48.157597+0000","last_became_peered":"2024-05-08T08:25:48.157597+0000","last_unstale":"2024-05-08T08:25:50.283008+0000","last_undegraded":"2024-05-08T08:25:50.283008+0000","last_fullsized":"2024-05-08T08:25:50.283008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:27.613891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080728699999999996,"stat_sum":{"num_bytes":45666384,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":2200,"num_read_kb":18882,"num_write":314,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"56'182","reported_seq":1010,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.152417+0000","last_change":"2024-05-08T08:25:47.152061+0000","last_active":"2024-05-08T08:25:47.152061+0000","last_peered":"2024-05-08T08:25:46.720808+0000","last_clean":"2024-05-08T08:25:18.944546+0000","last_became_active":"2024-05-08T08:25:46.258728+0000","last_became_peered":"2024-05-08T08:25:46.258728+0000","last_unstale":"2024-05-08T08:25:47.152417+0000","last_undegraded":"2024-05-08T08:25:47.152417+0000","last_fullsized":"2024-05-08T08:25:47.152417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040135600000000001,"stat_sum":{"num_bytes":17597145,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":1190,"num_read_kb":4735,"num_write":186,"num_write_kb":6959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'219","reported_seq":787,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.313207+0000","last_change":"2024-05-08T08:25:06.610175+0000","last_active":"2024-05-08T08:25:50.313207+0000","last_peered":"2024-05-08T08:25:50.313207+0000","last_clean":"2024-05-08T08:25:50.313207+0000","last_became_active":"2024-05-08T08:25:04.609185+0000","last_became_peered":"2024-05-08T08:25:04.609185+0000","last_unstale":"2024-05-08T08:25:50.313207+0000","last_undegraded":"2024-05-08T08:25:50.313207+0000","last_fullsized":"2024-05-08T08:25:50.313207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:39:14.749019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505687,"stat_sum":{"num_bytes":11945945,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":1053,"num_read_kb":13308,"num_write":232,"num_write_kb":12783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'568","reported_seq":1337,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.247719+0000","last_change":"2024-05-08T08:25:06.628387+0000","last_active":"2024-05-08T08:25:50.247719+0000","last_peered":"2024-05-08T08:25:50.247719+0000","last_clean":"2024-05-08T08:25:50.247719+0000","last_became_active":"2024-05-08T08:25:04.610818+0000","last_became_peered":"2024-05-08T08:25:04.610818+0000","last_unstale":"2024-05-08T08:25:50.247719+0000","last_undegraded":"2024-05-08T08:25:50.247719+0000","last_fullsized":"2024-05-08T08:25:50.247719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:34.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069309899999999997,"stat_sum":{"num_bytes":30029466,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3146,"num_read_kb":6177,"num_write":879,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"60'185","reported_seq":1057,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.312388+0000","last_change":"2024-05-08T08:25:06.625296+0000","last_active":"2024-05-08T08:25:50.312388+0000","last_peered":"2024-05-08T08:25:50.312388+0000","last_clean":"2024-05-08T08:25:50.312388+0000","last_became_active":"2024-05-08T08:25:04.605228+0000","last_became_peered":"2024-05-08T08:25:04.605228+0000","last_unstale":"2024-05-08T08:25:50.312388+0000","last_undegraded":"2024-05-08T08:25:50.312388+0000","last_fullsized":"2024-05-08T08:25:50.312388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:48:10.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011464940000000001,"stat_sum":{"num_bytes":10913207,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":1511,"num_read_kb":12774,"num_write":163,"num_write_kb":7914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"60'239","reported_seq":1582,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239393+0000","last_change":"2024-05-08T08:25:06.796215+0000","last_active":"2024-05-08T08:25:50.239393+0000","last_peered":"2024-05-08T08:25:50.239393+0000","last_clean":"2024-05-08T08:25:50.239393+0000","last_became_active":"2024-05-08T08:25:04.606180+0000","last_became_peered":"2024-05-08T08:25:04.606180+0000","last_unstale":"2024-05-08T08:25:50.239393+0000","last_undegraded":"2024-05-08T08:25:50.239393+0000","last_fullsized":"2024-05-08T08:25:50.239393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:59.196483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027126400000000001,"stat_sum":{"num_bytes":30398012,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":2362,"num_read_kb":16681,"num_write":271,"num_write_kb":4648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"59'333","reported_seq":753,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174111+0000","last_change":"2024-05-08T08:25:06.615415+0000","last_active":"2024-05-08T08:25:50.174111+0000","last_peered":"2024-05-08T08:25:50.174111+0000","last_clean":"2024-05-08T08:25:50.174111+0000","last_became_active":"2024-05-08T08:25:04.614760+0000","last_became_peered":"2024-05-08T08:25:04.614760+0000","last_unstale":"2024-05-08T08:25:50.174111+0000","last_undegraded":"2024-05-08T08:25:50.174111+0000","last_fullsized":"2024-05-08T08:25:50.174111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:51.380106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034964000000000002,"stat_sum":{"num_bytes":15590732,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1270,"num_read_kb":6723,"num_write":478,"num_write_kb":5875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.175991+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:25:50.175991+0000","last_peered":"2024-05-08T08:25:50.175991+0000","last_clean":"2024-05-08T08:25:50.175991+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:25:50.175991+0000","last_undegraded":"2024-05-08T08:25:50.175991+0000","last_fullsized":"2024-05-08T08:25:50.175991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:32.338505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"60'200","reported_seq":697,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.449596+0000","last_change":"2024-05-08T08:25:06.625323+0000","last_active":"2024-05-08T08:25:50.449596+0000","last_peered":"2024-05-08T08:25:50.449596+0000","last_clean":"2024-05-08T08:25:50.449596+0000","last_became_active":"2024-05-08T08:25:04.606855+0000","last_became_peered":"2024-05-08T08:25:04.606855+0000","last_unstale":"2024-05-08T08:25:50.449596+0000","last_undegraded":"2024-05-08T08:25:50.449596+0000","last_fullsized":"2024-05-08T08:25:50.449596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:16.237935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011708910000000001,"stat_sum":{"num_bytes":16728566,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":527,"num_read_kb":6360,"num_write":202,"num_write_kb":8951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.176035+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:25:50.176035+0000","last_peered":"2024-05-08T08:25:50.176035+0000","last_clean":"2024-05-08T08:25:50.176035+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:25:50.176035+0000","last_undegraded":"2024-05-08T08:25:50.176035+0000","last_fullsized":"2024-05-08T08:25:50.176035+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:54.867771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"52'247","reported_seq":1489,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.151756+0000","last_change":"2024-05-08T08:25:47.151756+0000","last_active":"2024-05-08T08:25:45.225592+0000","last_peered":"2024-05-08T08:25:43.120530+0000","last_clean":"2024-05-08T08:25:18.938588+0000","last_became_active":"2024-05-08T08:25:21.002993+0000","last_became_peered":"2024-05-08T08:25:21.002993+0000","last_unstale":"2024-05-08T08:25:47.151756+0000","last_undegraded":"2024-05-08T08:25:47.151756+0000","last_fullsized":"2024-05-08T08:25:47.151756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847699999999999,"stat_sum":{"num_bytes":36763536,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2124,"num_read_kb":25312,"num_write":253,"num_write_kb":4436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174067+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:25:50.174067+0000","last_peered":"2024-05-08T08:25:50.174067+0000","last_clean":"2024-05-08T08:25:50.174067+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:25:50.174067+0000","last_undegraded":"2024-05-08T08:25:50.174067+0000","last_fullsized":"2024-05-08T08:25:50.174067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:43:18.265730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"60'597","reported_seq":2813,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.316168+0000","last_change":"2024-05-08T08:25:06.611208+0000","last_active":"2024-05-08T08:25:50.316168+0000","last_peered":"2024-05-08T08:25:50.316168+0000","last_clean":"2024-05-08T08:25:50.316168+0000","last_became_active":"2024-05-08T08:25:04.608555+0000","last_became_peered":"2024-05-08T08:25:04.608555+0000","last_unstale":"2024-05-08T08:25:50.316168+0000","last_undegraded":"2024-05-08T08:25:50.316168+0000","last_fullsized":"2024-05-08T08:25:50.316168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:04.768564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195331,"stat_sum":{"num_bytes":44612987,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":17,"num_read":5039,"num_read_kb":59082,"num_write":858,"num_write_kb":32480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.206174+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:25:50.206174+0000","last_peered":"2024-05-08T08:25:50.206174+0000","last_clean":"2024-05-08T08:25:50.206174+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:25:50.206174+0000","last_undegraded":"2024-05-08T08:25:50.206174+0000","last_fullsized":"2024-05-08T08:25:50.206174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"59'237","reported_seq":1720,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.318487+0000","last_change":"2024-05-08T08:25:48.159785+0000","last_active":"2024-05-08T08:25:50.318487+0000","last_peered":"2024-05-08T08:25:50.318487+0000","last_clean":"2024-05-08T08:25:50.318487+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:25:50.318487+0000","last_undegraded":"2024-05-08T08:25:50.318487+0000","last_fullsized":"2024-05-08T08:25:50.318487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:07.930167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062277100000000005,"stat_sum":{"num_bytes":35575901,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":2222,"num_read_kb":35423,"num_write":257,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'378","reported_seq":1409,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.171762+0000","last_change":"2024-05-08T08:25:06.628002+0000","last_active":"2024-05-08T08:25:50.171762+0000","last_peered":"2024-05-08T08:25:50.171762+0000","last_clean":"2024-05-08T08:25:50.171762+0000","last_became_active":"2024-05-08T08:25:04.615077+0000","last_became_peered":"2024-05-08T08:25:04.615077+0000","last_unstale":"2024-05-08T08:25:50.171762+0000","last_undegraded":"2024-05-08T08:25:50.171762+0000","last_fullsized":"2024-05-08T08:25:50.171762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.908263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057249000000000004,"stat_sum":{"num_bytes":11737820,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":2433,"num_read_kb":9657,"num_write":442,"num_write_kb":6631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":255,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238459+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:25:50.238459+0000","last_peered":"2024-05-08T08:25:50.238459+0000","last_clean":"2024-05-08T08:25:50.238459+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:25:50.238459+0000","last_undegraded":"2024-05-08T08:25:50.238459+0000","last_fullsized":"2024-05-08T08:25:50.238459+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:25:47.151317+0000","last_change":"2024-05-08T08:25:45.225724+0000","last_active":"2024-05-08T08:25:47.151317+0000","last_peered":"2024-05-08T08:25:47.151317+0000","last_clean":"2024-05-08T08:25:47.151317+0000","last_became_active":"2024-05-08T08:25:45.225388+0000","last_became_peered":"2024-05-08T08:25:45.225388+0000","last_unstale":"2024-05-08T08:25:47.151317+0000","last_undegraded":"2024-05-08T08:25:47.151317+0000","last_fullsized":"2024-05-08T08:25:47.151317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:54.551477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"56'185","reported_seq":883,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.154031+0000","last_change":"2024-05-08T08:25:47.151090+0000","last_active":"2024-05-08T08:25:47.151090+0000","last_peered":"2024-05-08T08:25:46.732061+0000","last_clean":"2024-05-08T08:25:18.936967+0000","last_became_active":"2024-05-08T08:25:46.260405+0000","last_became_peered":"2024-05-08T08:25:46.260405+0000","last_unstale":"2024-05-08T08:25:47.154031+0000","last_undegraded":"2024-05-08T08:25:47.154031+0000","last_fullsized":"2024-05-08T08:25:47.154031+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034192400000000001,"stat_sum":{"num_bytes":23073517,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":580,"num_read_kb":8154,"num_write":223,"num_write_kb":8181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238782+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:25:50.238782+0000","last_peered":"2024-05-08T08:25:50.238782+0000","last_clean":"2024-05-08T08:25:50.238782+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:25:50.238782+0000","last_undegraded":"2024-05-08T08:25:50.238782+0000","last_fullsized":"2024-05-08T08:25:50.238782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"60'210","reported_seq":805,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.314514+0000","last_change":"2024-05-08T08:25:06.615595+0000","last_active":"2024-05-08T08:25:50.314514+0000","last_peered":"2024-05-08T08:25:50.314514+0000","last_clean":"2024-05-08T08:25:50.314514+0000","last_became_active":"2024-05-08T08:25:04.611900+0000","last_became_peered":"2024-05-08T08:25:04.611900+0000","last_unstale":"2024-05-08T08:25:50.314514+0000","last_undegraded":"2024-05-08T08:25:50.314514+0000","last_fullsized":"2024-05-08T08:25:50.314514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:15:47.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050119499999999996,"stat_sum":{"num_bytes":19631352,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":533,"num_read_kb":20098,"num_write":219,"num_write_kb":15522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174391+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:25:50.174391+0000","last_peered":"2024-05-08T08:25:50.174391+0000","last_clean":"2024-05-08T08:25:50.174391+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:25:50.174391+0000","last_undegraded":"2024-05-08T08:25:50.174391+0000","last_fullsized":"2024-05-08T08:25:50.174391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"59'149","reported_seq":1103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.315072+0000","last_change":"2024-05-08T08:25:06.610137+0000","last_active":"2024-05-08T08:25:50.315072+0000","last_peered":"2024-05-08T08:25:50.315072+0000","last_clean":"2024-05-08T08:25:50.315072+0000","last_became_active":"2024-05-08T08:25:04.606801+0000","last_became_peered":"2024-05-08T08:25:04.606801+0000","last_unstale":"2024-05-08T08:25:50.315072+0000","last_undegraded":"2024-05-08T08:25:50.315072+0000","last_fullsized":"2024-05-08T08:25:50.315072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:04.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042592699999999999,"stat_sum":{"num_bytes":61920517,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":899,"num_read_kb":46926,"num_write":151,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238584+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:25:50.238584+0000","last_peered":"2024-05-08T08:25:50.238584+0000","last_clean":"2024-05-08T08:25:50.238584+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:25:50.238584+0000","last_undegraded":"2024-05-08T08:25:50.238584+0000","last_fullsized":"2024-05-08T08:25:50.238584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"56'216","reported_seq":2169,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.206318+0000","last_change":"2024-05-08T08:25:48.159093+0000","last_active":"2024-05-08T08:25:50.206318+0000","last_peered":"2024-05-08T08:25:50.206318+0000","last_clean":"2024-05-08T08:25:50.206318+0000","last_became_active":"2024-05-08T08:25:48.158198+0000","last_became_peered":"2024-05-08T08:25:48.158198+0000","last_unstale":"2024-05-08T08:25:50.206318+0000","last_undegraded":"2024-05-08T08:25:50.206318+0000","last_fullsized":"2024-05-08T08:25:50.206318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.689664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001097338,"stat_sum":{"num_bytes":23653360,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3084,"num_read_kb":21242,"num_write":250,"num_write_kb":16653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"59'218","reported_seq":1078,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174056+0000","last_change":"2024-05-08T08:25:06.615379+0000","last_active":"2024-05-08T08:25:50.174056+0000","last_peered":"2024-05-08T08:25:50.174056+0000","last_clean":"2024-05-08T08:25:50.174056+0000","last_became_active":"2024-05-08T08:25:04.901435+0000","last_became_peered":"2024-05-08T08:25:04.901435+0000","last_unstale":"2024-05-08T08:25:50.174056+0000","last_undegraded":"2024-05-08T08:25:50.174056+0000","last_fullsized":"2024-05-08T08:25:50.174056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:02.472778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039550799999999998,"stat_sum":{"num_bytes":11085840,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2089,"num_read_kb":6401,"num_write":209,"num_write_kb":3268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"60'366","reported_seq":824,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.313079+0000","last_change":"2024-05-08T08:25:06.612336+0000","last_active":"2024-05-08T08:25:50.313079+0000","last_peered":"2024-05-08T08:25:50.313079+0000","last_clean":"2024-05-08T08:25:50.313079+0000","last_became_active":"2024-05-08T08:25:04.615227+0000","last_became_peered":"2024-05-08T08:25:04.615227+0000","last_unstale":"2024-05-08T08:25:50.313079+0000","last_undegraded":"2024-05-08T08:25:50.313079+0000","last_fullsized":"2024-05-08T08:25:50.313079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:23.233411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034968800000000002,"stat_sum":{"num_bytes":28282536,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":1467,"num_read_kb":16390,"num_write":505,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"58'176","reported_seq":1328,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.312050+0000","last_change":"2024-05-08T08:25:06.610868+0000","last_active":"2024-05-08T08:25:50.312050+0000","last_peered":"2024-05-08T08:25:50.312050+0000","last_clean":"2024-05-08T08:25:50.312050+0000","last_became_active":"2024-05-08T08:25:04.608456+0000","last_became_peered":"2024-05-08T08:25:04.608456+0000","last_unstale":"2024-05-08T08:25:50.312050+0000","last_undegraded":"2024-05-08T08:25:50.312050+0000","last_fullsized":"2024-05-08T08:25:50.312050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:37.416621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112149,"stat_sum":{"num_bytes":34120901,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":1096,"num_read_kb":14290,"num_write":176,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"59'192","reported_seq":864,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174647+0000","last_change":"2024-05-08T08:25:06.624428+0000","last_active":"2024-05-08T08:25:50.174647+0000","last_peered":"2024-05-08T08:25:50.174647+0000","last_clean":"2024-05-08T08:25:50.174647+0000","last_became_active":"2024-05-08T08:25:04.606925+0000","last_became_peered":"2024-05-08T08:25:04.606925+0000","last_unstale":"2024-05-08T08:25:50.174647+0000","last_undegraded":"2024-05-08T08:25:50.174647+0000","last_fullsized":"2024-05-08T08:25:50.174647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:24.692160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046376499999999999,"stat_sum":{"num_bytes":58995571,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":614,"num_read_kb":34697,"num_write":264,"num_write_kb":22805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'177","reported_seq":468,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.315221+0000","last_change":"2024-05-08T08:25:06.796419+0000","last_active":"2024-05-08T08:25:50.315221+0000","last_peered":"2024-05-08T08:25:50.315221+0000","last_clean":"2024-05-08T08:25:50.315221+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:25:50.315221+0000","last_undegraded":"2024-05-08T08:25:50.315221+0000","last_fullsized":"2024-05-08T08:25:50.315221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:45.240770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046517299999999999,"stat_sum":{"num_bytes":10995804,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":216,"num_read_kb":7926,"num_write":178,"num_write_kb":4861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"59'322","reported_seq":2341,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238601+0000","last_change":"2024-05-08T08:25:06.796635+0000","last_active":"2024-05-08T08:25:50.238601+0000","last_peered":"2024-05-08T08:25:50.238601+0000","last_clean":"2024-05-08T08:25:50.238601+0000","last_became_active":"2024-05-08T08:25:04.615004+0000","last_became_peered":"2024-05-08T08:25:04.615004+0000","last_unstale":"2024-05-08T08:25:50.238601+0000","last_undegraded":"2024-05-08T08:25:50.238601+0000","last_fullsized":"2024-05-08T08:25:50.238601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:04.429594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060116800000000001,"stat_sum":{"num_bytes":43995266,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":3394,"num_read_kb":37060,"num_write":333,"num_write_kb":22695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"59'213","reported_seq":728,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239137+0000","last_change":"2024-05-08T08:25:06.796737+0000","last_active":"2024-05-08T08:25:50.239137+0000","last_peered":"2024-05-08T08:25:50.239137+0000","last_clean":"2024-05-08T08:25:50.239137+0000","last_became_active":"2024-05-08T08:25:04.615942+0000","last_became_peered":"2024-05-08T08:25:04.615942+0000","last_unstale":"2024-05-08T08:25:50.239137+0000","last_undegraded":"2024-05-08T08:25:50.239137+0000","last_fullsized":"2024-05-08T08:25:50.239137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:54.886102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065917600000000003,"stat_sum":{"num_bytes":5724433,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":794,"num_read_kb":2629,"num_write":252,"num_write_kb":1884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"60'546","reported_seq":1135,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.278796+0000","last_change":"2024-05-08T08:25:06.612297+0000","last_active":"2024-05-08T08:25:50.278796+0000","last_peered":"2024-05-08T08:25:50.278796+0000","last_clean":"2024-05-08T08:25:50.278796+0000","last_became_active":"2024-05-08T08:25:04.616212+0000","last_became_peered":"2024-05-08T08:25:04.616212+0000","last_unstale":"2024-05-08T08:25:50.278796+0000","last_undegraded":"2024-05-08T08:25:50.278796+0000","last_fullsized":"2024-05-08T08:25:50.278796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:31.116080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031704699999999999,"stat_sum":{"num_bytes":5284092,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":2537,"num_read_kb":6619,"num_write":839,"num_write_kb":3465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"59'204","reported_seq":933,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239287+0000","last_change":"2024-05-08T08:25:06.796231+0000","last_active":"2024-05-08T08:25:50.239287+0000","last_peered":"2024-05-08T08:25:50.239287+0000","last_clean":"2024-05-08T08:25:50.239287+0000","last_became_active":"2024-05-08T08:25:04.615031+0000","last_became_peered":"2024-05-08T08:25:04.615031+0000","last_unstale":"2024-05-08T08:25:50.239287+0000","last_undegraded":"2024-05-08T08:25:50.239287+0000","last_fullsized":"2024-05-08T08:25:50.239287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:30:30.412714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040064000000000001,"stat_sum":{"num_bytes":23621913,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":829,"num_read_kb":19159,"num_write":222,"num_write_kb":5194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"58'253","reported_seq":1153,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.316670+0000","last_change":"2024-05-08T08:25:06.628036+0000","last_active":"2024-05-08T08:25:50.316670+0000","last_peered":"2024-05-08T08:25:50.316670+0000","last_clean":"2024-05-08T08:25:50.316670+0000","last_became_active":"2024-05-08T08:25:04.609651+0000","last_became_peered":"2024-05-08T08:25:04.609651+0000","last_unstale":"2024-05-08T08:25:50.316670+0000","last_undegraded":"2024-05-08T08:25:50.316670+0000","last_fullsized":"2024-05-08T08:25:50.316670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:30.467433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050895200000000002,"stat_sum":{"num_bytes":20883517,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":1350,"num_read_kb":20411,"num_write":319,"num_write_kb":13515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"60'514","reported_seq":1361,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.240943+0000","last_change":"2024-05-08T08:25:06.796497+0000","last_active":"2024-05-08T08:25:50.240943+0000","last_peered":"2024-05-08T08:25:50.240943+0000","last_clean":"2024-05-08T08:25:50.240943+0000","last_became_active":"2024-05-08T08:25:04.616040+0000","last_became_peered":"2024-05-08T08:25:04.616040+0000","last_unstale":"2024-05-08T08:25:50.240943+0000","last_undegraded":"2024-05-08T08:25:50.240943+0000","last_fullsized":"2024-05-08T08:25:50.240943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:51.767601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050921500000000004,"stat_sum":{"num_bytes":48287375,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":2748,"num_read_kb":15447,"num_write":830,"num_write_kb":12206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":894868603,"num_objects":1122,"num_object_clones":261,"num_object_copies":2244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1122,"num_whiteouts":462,"num_read":60162,"num_read_kb":610107,"num_write":12072,"num_write_kb":350973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45748224,"data_stored":518901662,"data_compressed":16329872,"data_compressed_allocated":39673856,"data_compressed_original":515224942,"omap_allocated":9107,"internal_metadata":0},"log_size":9435,"ondisk_log_size":9435,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47032,"kb_used_data":7528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324808,"statfs":{"total":96636764160,"available":96588603392,"internally_reserved":0,"allocated":7708672,"data_stored":85619922,"data_compressed":2756454,"data_compressed_allocated":6094848,"data_compressed_original":84713283,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":2551808,"data_stored":21471733,"data_compressed":615042,"data_compressed_allocated":1548288,"data_compressed_original":21004288,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43352,"kb_used_data":8016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328488,"statfs":{"total":96636764160,"available":96592371712,"internally_reserved":0,"allocated":8208384,"data_stored":85986856,"data_compressed":2559614,"data_compressed_allocated":6623232,"data_compressed_original":85109415,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53140,"kb_used_data":8072,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318700,"statfs":{"total":96636764160,"available":96582348800,"internally_reserved":0,"allocated":8265728,"data_stored":85814967,"data_compressed":2840977,"data_compressed_allocated":6426624,"data_compressed_original":84793846,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45168,"kb_used_data":9816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326672,"statfs":{"total":96636764160,"available":96590512128,"internally_reserved":0,"allocated":10051584,"data_stored":103736403,"data_compressed":3214260,"data_compressed_allocated":8335360,"data_compressed_original":102821368,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33500,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338340,"statfs":{"total":96636764160,"available":96602460160,"internally_reserved":0,"allocated":2375680,"data_stored":16593268,"data_compressed":721391,"data_compressed_allocated":1368064,"data_compressed_original":16076703,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":54,"seq":231928233986,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33676,"kb_used_data":4992,"kb_used_omap":13,"kb_used_meta":28658,"kb_avail":94338164,"statfs":{"total":96636764160,"available":96602279936,"internally_reserved":0,"allocated":5111808,"data_stored":46096708,"data_compressed":1232659,"data_compressed_allocated":3903488,"data_compressed_original":45473792,"omap_allocated":13550,"internal_metadata":29346578},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68588,"kb_used_data":6884,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94303252,"statfs":{"total":96636764160,"available":96566530048,"internally_reserved":0,"allocated":7049216,"data_stored":77230786,"data_compressed":2399361,"data_compressed_allocated":5447680,"data_compressed_original":76379127,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6369280,"data_stored":76922333,"data_compressed":2399361,"data_compressed_allocated":5447680,"data_compressed_original":76379127,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4395008,"data_stored":45209515,"data_compressed":1227716,"data_compressed_allocated":3866624,"data_compressed_original":44900352,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1679360,"data_stored":16278634,"data_compressed":721391,"data_compressed_allocated":1368064,"data_compressed_original":16076703,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9375744,"data_stored":103428184,"data_compressed":3214260,"data_compressed_allocated":8335360,"data_compressed_original":102821368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7581696,"data_stored":85506484,"data_compressed":2840977,"data_compressed_allocated":6426624,"data_compressed_original":84793846,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7524352,"data_stored":85678342,"data_compressed":2559614,"data_compressed_allocated":6623232,"data_compressed_original":85109415,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":21157099,"data_compressed":615042,"data_compressed_allocated":1548288,"data_compressed_original":21004288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6967296,"data_stored":84721071,"data_compressed":2751511,"data_compressed_allocated":6057984,"data_compressed_original":84139843,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:52.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-05-08T08:25:52.110 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:52.117+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:52.175 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:25:52.192 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:52.192 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:52.192 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:52.192 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:52.192 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:52.201 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:52.213 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:52.220+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:52.311 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:52.319+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:52.403 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:52.403 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:52.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:52.403 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 54.843785629999999, 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.409 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14412 3.d 3:b3426b33:::rbd_data.119174045df.0000000000000019:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 27.385174969000001, 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024946593, 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14412 2024-05-08T08:25:52.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024217+0000", 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024218+0000", 2024-05-08T08:25:52.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7999999999999995e-07 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024221+0000", 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5489999999999998e-06 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024231+0000", 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.611e-06 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024236+0000", 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1950000000000002e-06 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024278+0000", 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2191000000000003e-05 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.024328+0000", 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0008e-05 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:52.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.028903+0000", 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0045743340000000002 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049023+0000", 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020120815 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049106+0000", 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2900999999999998e-05 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:52.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049140+0000", 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4029000000000002e-05 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.049164+0000", 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3280000000000001e-05 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23287 3.13 3:ced44c57:::rbd_data.1191ea0f1b1f.0000000000000006:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 21.896332904000001, 2024-05-08T08:25:52.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020994202999999999, 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23287 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513059+0000", 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.513061+0000", 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5889999999999999e-06 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524070+0000", 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.011009694 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524089+0000", 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8819000000000001e-05 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524099+0000", 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9580000000000002e-06 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524157+0000", 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8106000000000002e-05 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524271+0000", 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000113231 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.524698+0000", 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00042707900000000002 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533002+0000", 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083041999999999994 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533940+0000", 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00093861900000000002 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534001+0000", 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0492999999999998e-05 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.534053+0000", 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2414999999999999e-05 2024-05-08T08:25:52.420 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:31360 3.f 3:f2e41b89:::rbd_data.119198308290.0000000000000036:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e52)", 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3101959739999991, 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020342331000000002, 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 31360 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099196+0000", 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099197+0000", 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.085e-06 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099202+0000", 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.78e-06 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099214+0000", 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2369e-05 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099223+0000", 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6649999999999992e-06 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099283+0000", 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9784000000000002e-05 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.099366+0000", 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3091999999999997e-05 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 4", 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.103600+0000", 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0042338469999999998 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119405+0000", 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.015805146999999999 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119423+0000", 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8283000000000001e-05 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:52.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119482+0000", 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9206999999999997e-05 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:43.119538+0000", 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6072000000000001e-05 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50131 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901513+0000", 2024-05-08T08:25:52.426 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50787956300000003, 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018369307000000001, 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50131 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901513+0000", 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901513+0000", 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901513+0000", 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4400000000000001e-07 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901515+0000", 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.902e-06 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901521+0000", 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.263e-06 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901526+0000", 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.746e-06 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.918572+0000", 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.017046288 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919732+0000", 2024-05-08T08:25:52.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001160229 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919882+0000", 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00014943500000000001 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50132 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901531+0000", 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50786103999999999, 2024-05-08T08:25:52.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018577739999999999, 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50132 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901531+0000", 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901531+0000", 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901531+0000", 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2099999999999998e-07 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901533+0000", 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7999999999999999e-06 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901539+0000", 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6559999999999997e-06 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901544+0000", 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7160000000000002e-06 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919928+0000", 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018384000000000001 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920011+0000", 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3238999999999994e-05 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920109+0000", 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8008000000000001e-05 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50133 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 138240~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.434 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901549+0000", 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50784351599999999, 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018969389999999999, 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50133 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901549+0000", 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901549+0000", 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901549+0000", 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901551+0000", 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7990000000000001e-06 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901556+0000", 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6629999999999998e-06 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901561+0000", 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6959999999999996e-06 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920321+0000", 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018759997 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920414+0000", 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3189999999999994e-05 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920518+0000", 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000103686 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50134 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901566+0000", 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50782584900000005, 2024-05-08T08:25:52.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019183865000000001, 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50134 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901566+0000", 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.439 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901566+0000", 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901567+0000", 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901568+0000", 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.776e-06 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901574+0000", 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6300000000000003e-06 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901579+0000", 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5940000000000004e-06 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920564+0000", 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.018985463000000001 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920655+0000", 2024-05-08T08:25:52.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.145e-05 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920750+0000", 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4633e-05 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50135 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 156672~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901585+0000", 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50780749400000003, 2024-05-08T08:25:52.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019891109000000001, 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50135 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901585+0000", 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901585+0000", 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901585+0000", 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4299999999999999e-07 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901587+0000", 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.995e-06 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901593+0000", 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2709999999999998e-06 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901598+0000", 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.967e-06 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920804+0000", 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019206274999999998 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920883+0000", 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8571000000000004e-05 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921476+0000", 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00059268700000000005 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50136 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 165888~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901609+0000", 2024-05-08T08:25:52.446 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50778337699999998, 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020118186, 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50136 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901609+0000", 2024-05-08T08:25:52.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901609+0000", 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901609+0000", 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1699999999999999e-07 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901612+0000", 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.475e-06 2024-05-08T08:25:52.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901618+0000", 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2559999999999999e-06 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901629+0000", 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1082e-05 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921526+0000", 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019897056 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921611+0000", 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5227000000000002e-05 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921727+0000", 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000115673 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50137 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 175104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901638+0000", 2024-05-08T08:25:52.450 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50775428700000003, 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020296313, 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50137 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901638+0000", 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901638+0000", 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901638+0000", 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901641+0000", 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3949999999999999e-06 2024-05-08T08:25:52.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901652+0000", 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1822e-05 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901658+0000", 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2100000000000001e-06 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921757+0000", 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020099357000000002 2024-05-08T08:25:52.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921837+0000", 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9643999999999998e-05 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921934+0000", 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7443999999999997e-05 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50138 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 184320~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901672+0000", 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50771996900000005, 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020513080999999999, 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50138 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901672+0000", 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901672+0000", 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901673+0000", 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6000000000000004e-07 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901675+0000", 2024-05-08T08:25:52.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.723e-06 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901682+0000", 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5780000000000004e-06 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901687+0000", 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0590000000000002e-06 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921976+0000", 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020288528 2024-05-08T08:25:52.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922070+0000", 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4769e-05 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922185+0000", 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114864 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.458 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50139 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 193536~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901693+0000", 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50769926300000001, 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020716795999999999, 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50139 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901693+0000", 2024-05-08T08:25:52.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901693+0000", 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901693+0000", 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1300000000000001e-07 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901695+0000", 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.79e-06 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901701+0000", 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.694e-06 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901705+0000", 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922221+0000", 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020515532999999999 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922306+0000", 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4858000000000006e-05 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922410+0000", 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000104011 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50140 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 202752~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901710+0000", 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50768168700000005, 2024-05-08T08:25:52.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020925521999999998, 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50140 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901710+0000", 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901710+0000", 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901711+0000", 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901713+0000", 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0080000000000001e-06 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901718+0000", 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5929999999999996e-06 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901723+0000", 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.775e-06 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922440+0000", 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020716868999999999 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922524+0000", 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4314000000000006e-05 2024-05-08T08:25:52.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922636+0000", 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000111554 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50141 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 211968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901728+0000", 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50766367199999995, 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021130716000000001, 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.466 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50141 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901728+0000", 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901728+0000", 2024-05-08T08:25:52.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901729+0000", 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901731+0000", 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.81e-06 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901736+0000", 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2549999999999997e-06 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901741+0000", 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7439999999999998e-06 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922667+0000", 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020926561 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922756+0000", 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8936000000000004e-05 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922859+0000", 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010306000000000001 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50142 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 221184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901747+0000", 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50764522899999998, 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021336569999999999, 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50142 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901747+0000", 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901747+0000", 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901747+0000", 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901749+0000", 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.973e-06 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901755+0000", 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0669999999999997e-06 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901760+0000", 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.651e-06 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922900+0000", 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021140333000000001 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.922985+0000", 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4552e-05 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923083+0000", 2024-05-08T08:25:52.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8667999999999998e-05 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50143 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 230400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901772+0000", 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50761988300000005, 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021516080999999999, 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50143 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901772+0000", 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901772+0000", 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901773+0000", 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901775+0000", 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3779999999999999e-06 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901781+0000", 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2840000000000003e-06 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901787+0000", 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4060000000000004e-06 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923112+0000", 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02132539 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923192+0000", 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9633000000000001e-05 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923288+0000", 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6578999999999998e-05 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50144 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 239616~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901793+0000", 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50759884, 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021695703, 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.478 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50144 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901793+0000", 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901793+0000", 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901794+0000", 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6699999999999999e-07 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901804+0000", 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0722e-05 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901822+0000", 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7632e-05 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901832+0000", 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0277e-05 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923316+0000", 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021483944000000001 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923392+0000", 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6271000000000002e-05 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923489+0000", 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6490000000000006e-05 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50145 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 248832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901840+0000", 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50755176800000001, 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021854054000000001, 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.482 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50145 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901840+0000", 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901840+0000", 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901841+0000", 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901843+0000", 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1830000000000001e-06 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901849+0000", 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9830000000000001e-06 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901854+0000", 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7949999999999998e-06 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923516+0000", 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021662711000000001 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923593+0000", 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6248999999999995e-05 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923694+0000", 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010170699999999999 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50146 3.13 3:c81ef58a:::rbd_data.11914ad0077c.0000000000000008:1a [read 258048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.486 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.901863+0000", 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.50752936100000001, 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022080643, 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50146 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:52.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901863+0000", 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901863+0000", 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901863+0000", 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4999999999999998e-07 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:52.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901865+0000", 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6950000000000001e-06 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901871+0000", 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2920000000000001e-06 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.901876+0000", 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7400000000000004e-06 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:52.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923737+0000", 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021860628 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923832+0000", 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5699999999999995e-05 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923943+0000", 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00011113800000000001 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.490 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.491 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:52.491 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:52.491 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:52.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:52.493 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:52.500+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:52.580 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:52.580 INFO:teuthology.orchestra.run.smithi031.stdout:{"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":68,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":62,"num_osds":8,"num_up_osds":8,"osd_up_since":1715156744,"num_in_osds":8,"osd_in_since":1715156686,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":35},{"state_name":"remapped+peering","count":3},{"state_name":"peering","count":3}],"num_pgs":41,"num_pools":3,"num_objects":1031,"data_bytes":822200526,"bytes_used":353935360,"bytes_avail":772740177920,"bytes_total":773094113280,"inactive_pgs_ratio":0.14634145796298981,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":1133701,"recovering_keys_per_sec":2,"num_objects_recovered":83,"num_bytes_recovered":4535355,"num_keys_recovered":12,"read_bytes_sec":0,"write_bytes_sec":4095,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-05-08T08:25:50.430029+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"3":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"4":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"5":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"6":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"7":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-05-08T08:25:52.593 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:52.600+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:52.594 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-05-08T08:25:52.594 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:52.614 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:52.614 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:52.614 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:52.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:52.707 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:52.715+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:52.795 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:52.802+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:52.812 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:52.812 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:52.812 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:52.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:52.822 INFO:teuthology.orchestra.run.smithi031.stdout:3962 write 0x107844a thru 0x10788ef (0x4a6 bytes) 2024-05-08T08:25:52.825 INFO:teuthology.orchestra.run.smithi031.stdout:3963 punch from 0x51422a9 to 0x51483df, (0x6136 bytes) 2024-05-08T08:25:52.825 INFO:teuthology.orchestra.run.smithi031.stdout:3964 writesame 0x20b447e thru 0x20b924e data_size 0x1(0x4dd1 bytes) 2024-05-08T08:25:52.839 INFO:teuthology.orchestra.run.smithi031.stdout:3965 punch from 0x99cbe5c to 0x99d7497, (0xb63b bytes) 2024-05-08T08:25:52.840 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:52.840 INFO:teuthology.orchestra.run.smithi031.stdout:3967 writesame 0x9d237c1 thru 0x9d2588b data_size 0x17(0x20cb bytes) 2024-05-08T08:25:52.852 INFO:teuthology.orchestra.run.smithi031.stdout:3968 write 0x75efce4 thru 0x75fd3be (0xd6db bytes) 2024-05-08T08:25:52.864 INFO:teuthology.orchestra.run.smithi031.stdout:3969 write 0x25086c1 thru 0x2517760 (0xf0a0 bytes) 2024-05-08T08:25:52.877 INFO:teuthology.orchestra.run.smithi031.stdout:3972 write 0x1d114d6 thru 0x1d18fac (0x7ad7 bytes) 2024-05-08T08:25:52.886 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:52.887 INFO:teuthology.orchestra.run.smithi031.stdout:3974 writesame 0x335f0b4 thru 0x3360114 data_size 0x1(0x1061 bytes) 2024-05-08T08:25:52.896 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:52.903+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:52.908 INFO:teuthology.orchestra.run.smithi031.stdout:3976 trunc from 0xd92ded2 to 0x1fa846d 2024-05-08T08:25:52.932 INFO:teuthology.orchestra.run.smithi031.stdout:3977 write 0x35b407 thru 0x35bd1d (0x917 bytes) 2024-05-08T08:25:52.933 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:52.939 INFO:teuthology.orchestra.run.smithi031.stdout:3979 trunc from 0x1fa846d to 0x75d017c 2024-05-08T08:25:52.945 INFO:teuthology.orchestra.run.smithi031.stdout:3980 read 0x30a9b7e thru 0x30b01c3 (0x6646 bytes) 2024-05-08T08:25:52.945 INFO:teuthology.orchestra.run.smithi031.stdout:3981 write 0x73989 thru 0x8042f (0xcaa7 bytes) 2024-05-08T08:25:52.951 INFO:teuthology.orchestra.run.smithi031.stdout:3988 trunc from 0x75d017c to 0x89d0937 2024-05-08T08:25:52.958 INFO:teuthology.orchestra.run.smithi031.stdout:3989 trunc from 0x89d0937 to 0xa4e69f0 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50476 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 161792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0571416039999999, 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018840722000000001, 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50476 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915327+0000", 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915329+0000", 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915336+0000", 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4880000000000001e-06 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915342+0000", 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.933988+0000", 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018646593 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934072+0000", 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3626000000000002e-05 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934167+0000", 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5087999999999995e-05 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50477 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 171008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.057119132, 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019055283999999999, 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50477 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915351+0000", 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.311e-06 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915363+0000", 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2031000000000001e-05 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915369+0000", 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6930000000000004e-06 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934223+0000", 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018853741 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934307+0000", 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4154999999999996e-05 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934404+0000", 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6973999999999997e-05 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50478 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 180224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:25:52.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.057090512, 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019257972000000002, 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50478 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915378+0000", 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915387+0000", 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999993e-06 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915396+0000", 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2499999999999995e-06 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915401+0000", 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.259e-06 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934434+0000", 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019032970999999999 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934535+0000", 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100479 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934635+0000", 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100445 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50479 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 189440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.978 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0570582530000001, 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019428513000000001, 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50479 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:52.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915413+0000", 2024-05-08T08:25:52.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915419+0000", 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7320000000000001e-06 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915424+0000", 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934664+0000", 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019239903999999999 2024-05-08T08:25:52.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934740+0000", 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5833000000000004e-05 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934838+0000", 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7832999999999997e-05 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50480 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 198656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.057036342, 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019651075, 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50480 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915432+0000", 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915434+0000", 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.221e-06 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915440+0000", 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4770000000000002e-06 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915444+0000", 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8330000000000001e-06 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934880+0000", 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019435555 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934984+0000", 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010403099999999999 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935083+0000", 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8566000000000005e-05 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50481 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0570164040000001, 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019849193000000001, 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50481 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915452+0000", 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915454+0000", 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3719999999999999e-06 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915461+0000", 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6290000000000004e-06 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915466+0000", 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:25:52.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935126+0000", 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019660074 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935221+0000", 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4598000000000005e-05 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935301+0000", 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.996e-05 2024-05-08T08:25:52.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50482 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0569948250000001, 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020069704000000001, 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50482 2024-05-08T08:25:52.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:25:52.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915476+0000", 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915482+0000", 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4030000000000004e-06 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915487+0000", 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9219999999999996e-06 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935343+0000", 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019856492999999999 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935433+0000", 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9485000000000001e-05 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935543+0000", 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109832 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50483 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 226304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0569744999999999, 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020296027000000001, 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50483 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.995 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915494+0000", 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915496+0000", 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915502+0000", 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7939999999999999e-06 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915507+0000", 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935582+0000", 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020075790999999999 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935672+0000", 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0074999999999994e-05 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935789+0000", 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011681299999999999 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.998 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50484 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0569546409999999, 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020571613999999998, 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50484 2024-05-08T08:25:52.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915514+0000", 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915516+0000", 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2809999999999998e-06 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915649+0000", 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000132776 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915668+0000", 2024-05-08T08:25:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9253e-05 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935846+0000", 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020177820999999999 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935950+0000", 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104478 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936085+0000", 2024-05-08T08:25:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000134618 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50485 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0567879549999999, 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020672174000000001, 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50485 2024-05-08T08:25:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4700000000000001e-07 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915684+0000", 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5820000000000002e-06 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915692+0000", 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1829999999999994e-06 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915706+0000", 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3747999999999999e-05 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936120+0000", 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020414472999999999 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936205+0000", 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4561000000000006e-05 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936352+0000", 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014698000000000001 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50486 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0567520770000001, 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020885167, 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50486 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915724+0000", 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4200000000000001e-06 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915734+0000", 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0796e-05 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915748+0000", 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3318e-05 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936388+0000", 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020640642000000001 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936491+0000", 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010300400000000001 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936601+0000", 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109523 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50737 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 208896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.048799169, 2024-05-08T08:25:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018673419, 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50737 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923672+0000", 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.252e-06 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923679+0000", 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923737+0000", 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8035000000000003e-05 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942158+0000", 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018421334000000001 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942240+0000", 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1409000000000006e-05 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942342+0000", 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102252 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50738 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:25:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.048715821, 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018832728, 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50738 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923753+0000", 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-07 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923756+0000", 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4510000000000001e-06 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923766+0000", 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.049e-05 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923773+0000", 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942387+0000", 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018614117999999999 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942475+0000", 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7874999999999995e-05 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942585+0000", 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011014300000000001 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50739 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0486508569999999, 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990561999999999, 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50739 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923828+0000", 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0988999999999999e-05 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923837+0000", 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6889999999999993e-06 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923851+0000", 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3528e-05 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942618+0000", 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018766998999999999 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942704+0000", 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5995000000000006e-05 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942807+0000", 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103837 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50740 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.0486074729999999, 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019168166, 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50740 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923861+0000", 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923863+0000", 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923871+0000", 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0569999999999993e-06 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923877+0000", 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942837+0000", 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018959999000000002 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942917+0000", 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0519000000000006e-05 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943028+0000", 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111402 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50741 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.048583625, 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019309865999999998, 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50741 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923891+0000", 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923894+0000", 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1829999999999998e-06 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923902+0000", 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0910000000000004e-06 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923907+0000", 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6729999999999997e-06 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943059+0000", 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019151695 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943123+0000", 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4098999999999997e-05 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943194+0000", 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0970999999999995e-05 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50742 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 1.048552701, 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019467485, 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50742 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923932+0000", 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6268000000000001e-05 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923939+0000", 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7270000000000007e-06 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923945+0000", 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5419999999999996e-06 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943232+0000", 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019287316999999998 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943315+0000", 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2854000000000003e-05 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943383+0000", 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7427000000000003e-05 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.81221371200000003, 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.81219261200000004, 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:25:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 0.81217195399999997, 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:25:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:25:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.048 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:53.049 INFO:teuthology.orchestra.run.smithi031.stdout:3990 trunc from 0xa4e69f0 to 0x3e64693 2024-05-08T08:25:53.050 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.050 INFO:teuthology.orchestra.run.smithi031.stdout:3992 punch from 0x255c968 to 0x255d50c, (0xba4 bytes) 2024-05-08T08:25:53.050 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:53.052 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:53.053 INFO:teuthology.orchestra.run.smithi031.stdout:3995 writesame 0xa2cfe1f thru 0xa2d3c44 data_size 0x4a(0x3e26 bytes) 2024-05-08T08:25:53.053 INFO:teuthology.orchestra.run.smithi031.stdout:3996 read 0xf66ea7 thru 0xf73f9e (0xd0f8 bytes) 2024-05-08T08:25:53.054 INFO:teuthology.orchestra.run.smithi031.stdout:3997 trunc from 0xa2d3c45 to 0x45f5846 2024-05-08T08:25:53.054 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-05-08T08:25:52.430333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":895458971,"num_objects":1124,"num_object_clones":261,"num_object_copies":2248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1124,"num_whiteouts":462,"num_read":60278,"num_read_kb":610207,"num_write":12231,"num_write_kb":353389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9552,"ondisk_log_size":9552,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":358144,"kb_used_data":50120,"kb_used_omap":24,"kb_used_meta":307879,"kb_avail":754616576,"statfs":{"total":773094113280,"available":772727373824,"internally_reserved":0,"allocated":51322880,"data_stored":522550643,"data_compressed":16339758,"data_compressed_allocated":39747584,"data_compressed_original":516371822,"omap_allocated":24704,"internal_metadata":315268992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":73258445,"num_objects":93,"num_object_clones":36,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":11,"num_read":8051,"num_read_kb":202107,"num_write":725,"num_write_kb":57118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":126,"num_bytes_recovered":9209885,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000592"},"pg_stats":[{"pgid":"3.1f","version":"60'292","reported_seq":1782,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.267575+0000","last_change":"2024-05-08T08:25:06.628835+0000","last_active":"2024-05-08T08:25:50.267575+0000","last_peered":"2024-05-08T08:25:50.267575+0000","last_clean":"2024-05-08T08:25:50.267575+0000","last_became_active":"2024-05-08T08:25:04.610836+0000","last_became_peered":"2024-05-08T08:25:04.610836+0000","last_unstale":"2024-05-08T08:25:50.267575+0000","last_undegraded":"2024-05-08T08:25:50.267575+0000","last_fullsized":"2024-05-08T08:25:50.267575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":292,"log_dups_size":0,"ondisk_log_size":292,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:51:12.849579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011177400000000001,"stat_sum":{"num_bytes":20581408,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":14,"num_read":3393,"num_read_kb":15646,"num_write":279,"num_write_kb":6170,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"60'218","reported_seq":1033,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.448840+0000","last_change":"2024-05-08T08:25:06.611278+0000","last_active":"2024-05-08T08:25:50.448840+0000","last_peered":"2024-05-08T08:25:50.448840+0000","last_clean":"2024-05-08T08:25:50.448840+0000","last_became_active":"2024-05-08T08:25:04.611994+0000","last_became_peered":"2024-05-08T08:25:04.611994+0000","last_unstale":"2024-05-08T08:25:50.448840+0000","last_undegraded":"2024-05-08T08:25:50.448840+0000","last_fullsized":"2024-05-08T08:25:50.448840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:12:31.242286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010168530000000001,"stat_sum":{"num_bytes":19775175,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":22,"num_read":997,"num_read_kb":21951,"num_write":262,"num_write_kb":15843,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"59'202","reported_seq":1102,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.314972+0000","last_change":"2024-05-08T08:25:06.625063+0000","last_active":"2024-05-08T08:25:50.314972+0000","last_peered":"2024-05-08T08:25:50.314972+0000","last_clean":"2024-05-08T08:25:50.314972+0000","last_became_active":"2024-05-08T08:25:04.610134+0000","last_became_peered":"2024-05-08T08:25:04.610134+0000","last_unstale":"2024-05-08T08:25:50.314972+0000","last_undegraded":"2024-05-08T08:25:50.314972+0000","last_fullsized":"2024-05-08T08:25:50.314972+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":202,"log_dups_size":0,"ondisk_log_size":202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:11.796747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005171,"stat_sum":{"num_bytes":14449780,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":1261,"num_read_kb":17820,"num_write":216,"num_write_kb":9850,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"60'207","reported_seq":1855,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.269796+0000","last_change":"2024-05-08T08:25:06.625044+0000","last_active":"2024-05-08T08:25:50.269796+0000","last_peered":"2024-05-08T08:25:50.269796+0000","last_clean":"2024-05-08T08:25:50.269796+0000","last_became_active":"2024-05-08T08:25:04.615120+0000","last_became_peered":"2024-05-08T08:25:04.615120+0000","last_unstale":"2024-05-08T08:25:50.269796+0000","last_undegraded":"2024-05-08T08:25:50.269796+0000","last_fullsized":"2024-05-08T08:25:50.269796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":207,"log_dups_size":0,"ondisk_log_size":207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:13:07.539902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010445210000000001,"stat_sum":{"num_bytes":94835931,"num_objects":31,"num_object_clones":7,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":12,"num_read":1976,"num_read_kb":56801,"num_write":265,"num_write_kb":21749,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"56'935","reported_seq":1478,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.172125+0000","last_change":"2024-05-08T08:25:06.628198+0000","last_active":"2024-05-08T08:25:50.172125+0000","last_peered":"2024-05-08T08:25:50.172125+0000","last_clean":"2024-05-08T08:25:50.172125+0000","last_became_active":"2024-05-08T08:25:04.612215+0000","last_became_peered":"2024-05-08T08:25:04.612215+0000","last_unstale":"2024-05-08T08:25:50.172125+0000","last_undegraded":"2024-05-08T08:25:50.172125+0000","last_fullsized":"2024-05-08T08:25:50.172125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":935,"log_dups_size":0,"ondisk_log_size":935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:22:26.847029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064530100000000003,"stat_sum":{"num_bytes":18110614,"num_objects":33,"num_object_clones":8,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":20,"num_read":5228,"num_read_kb":5326,"num_write":1545,"num_write_kb":2587,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"60'255","reported_seq":2320,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.283008+0000","last_change":"2024-05-08T08:25:48.158059+0000","last_active":"2024-05-08T08:25:50.283008+0000","last_peered":"2024-05-08T08:25:50.283008+0000","last_clean":"2024-05-08T08:25:50.283008+0000","last_became_active":"2024-05-08T08:25:48.157597+0000","last_became_peered":"2024-05-08T08:25:48.157597+0000","last_unstale":"2024-05-08T08:25:50.283008+0000","last_undegraded":"2024-05-08T08:25:50.283008+0000","last_fullsized":"2024-05-08T08:25:50.283008+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:54:27.613891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080728699999999996,"stat_sum":{"num_bytes":45666384,"num_objects":37,"num_object_clones":7,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":15,"num_read":2200,"num_read_kb":18882,"num_write":314,"num_write_kb":13818,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"56'182","reported_seq":1010,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.152417+0000","last_change":"2024-05-08T08:25:47.152061+0000","last_active":"2024-05-08T08:25:47.152061+0000","last_peered":"2024-05-08T08:25:46.720808+0000","last_clean":"2024-05-08T08:25:18.944546+0000","last_became_active":"2024-05-08T08:25:46.258728+0000","last_became_peered":"2024-05-08T08:25:46.258728+0000","last_unstale":"2024-05-08T08:25:47.152417+0000","last_undegraded":"2024-05-08T08:25:47.152417+0000","last_fullsized":"2024-05-08T08:25:47.152417+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040135600000000001,"stat_sum":{"num_bytes":17597145,"num_objects":30,"num_object_clones":6,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":1190,"num_read_kb":4735,"num_write":186,"num_write_kb":6959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"60'219","reported_seq":787,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.313207+0000","last_change":"2024-05-08T08:25:06.610175+0000","last_active":"2024-05-08T08:25:50.313207+0000","last_peered":"2024-05-08T08:25:50.313207+0000","last_clean":"2024-05-08T08:25:50.313207+0000","last_became_active":"2024-05-08T08:25:04.609185+0000","last_became_peered":"2024-05-08T08:25:04.609185+0000","last_unstale":"2024-05-08T08:25:50.313207+0000","last_undegraded":"2024-05-08T08:25:50.313207+0000","last_fullsized":"2024-05-08T08:25:50.313207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":219,"log_dups_size":0,"ondisk_log_size":219,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:39:14.749019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505687,"stat_sum":{"num_bytes":11945945,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":1053,"num_read_kb":13308,"num_write":232,"num_write_kb":12783,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"60'568","reported_seq":1337,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.247719+0000","last_change":"2024-05-08T08:25:06.628387+0000","last_active":"2024-05-08T08:25:50.247719+0000","last_peered":"2024-05-08T08:25:50.247719+0000","last_clean":"2024-05-08T08:25:50.247719+0000","last_became_active":"2024-05-08T08:25:04.610818+0000","last_became_peered":"2024-05-08T08:25:04.610818+0000","last_unstale":"2024-05-08T08:25:50.247719+0000","last_undegraded":"2024-05-08T08:25:50.247719+0000","last_fullsized":"2024-05-08T08:25:50.247719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":568,"log_dups_size":0,"ondisk_log_size":568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:26:34.821063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069309899999999997,"stat_sum":{"num_bytes":30029466,"num_objects":34,"num_object_clones":4,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":12,"num_read":3146,"num_read_kb":6177,"num_write":879,"num_write_kb":6878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"60'185","reported_seq":1057,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.312388+0000","last_change":"2024-05-08T08:25:06.625296+0000","last_active":"2024-05-08T08:25:50.312388+0000","last_peered":"2024-05-08T08:25:50.312388+0000","last_clean":"2024-05-08T08:25:50.312388+0000","last_became_active":"2024-05-08T08:25:04.605228+0000","last_became_peered":"2024-05-08T08:25:04.605228+0000","last_unstale":"2024-05-08T08:25:50.312388+0000","last_undegraded":"2024-05-08T08:25:50.312388+0000","last_fullsized":"2024-05-08T08:25:50.312388+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:48:10.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011464940000000001,"stat_sum":{"num_bytes":10913207,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":1511,"num_read_kb":12774,"num_write":163,"num_write_kb":7914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"60'239","reported_seq":1582,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239393+0000","last_change":"2024-05-08T08:25:06.796215+0000","last_active":"2024-05-08T08:25:50.239393+0000","last_peered":"2024-05-08T08:25:50.239393+0000","last_clean":"2024-05-08T08:25:50.239393+0000","last_became_active":"2024-05-08T08:25:04.606180+0000","last_became_peered":"2024-05-08T08:25:04.606180+0000","last_unstale":"2024-05-08T08:25:50.239393+0000","last_undegraded":"2024-05-08T08:25:50.239393+0000","last_fullsized":"2024-05-08T08:25:50.239393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":239,"log_dups_size":0,"ondisk_log_size":239,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:59.196483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027126400000000001,"stat_sum":{"num_bytes":30398012,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":2362,"num_read_kb":16681,"num_write":271,"num_write_kb":4648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"59'333","reported_seq":753,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174111+0000","last_change":"2024-05-08T08:25:06.615415+0000","last_active":"2024-05-08T08:25:50.174111+0000","last_peered":"2024-05-08T08:25:50.174111+0000","last_clean":"2024-05-08T08:25:50.174111+0000","last_became_active":"2024-05-08T08:25:04.614760+0000","last_became_peered":"2024-05-08T08:25:04.614760+0000","last_unstale":"2024-05-08T08:25:50.174111+0000","last_undegraded":"2024-05-08T08:25:50.174111+0000","last_fullsized":"2024-05-08T08:25:50.174111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":333,"log_dups_size":0,"ondisk_log_size":333,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:11:51.380106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034964000000000002,"stat_sum":{"num_bytes":15590732,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1270,"num_read_kb":6723,"num_write":478,"num_write_kb":5875,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.175991+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:25:50.175991+0000","last_peered":"2024-05-08T08:25:50.175991+0000","last_clean":"2024-05-08T08:25:50.175991+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:25:50.175991+0000","last_undegraded":"2024-05-08T08:25:50.175991+0000","last_fullsized":"2024-05-08T08:25:50.175991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:32.338505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"60'200","reported_seq":697,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.449596+0000","last_change":"2024-05-08T08:25:06.625323+0000","last_active":"2024-05-08T08:25:50.449596+0000","last_peered":"2024-05-08T08:25:50.449596+0000","last_clean":"2024-05-08T08:25:50.449596+0000","last_became_active":"2024-05-08T08:25:04.606855+0000","last_became_peered":"2024-05-08T08:25:04.606855+0000","last_unstale":"2024-05-08T08:25:50.449596+0000","last_undegraded":"2024-05-08T08:25:50.449596+0000","last_fullsized":"2024-05-08T08:25:50.449596+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":200,"log_dups_size":0,"ondisk_log_size":200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:16.237935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011708910000000001,"stat_sum":{"num_bytes":16728566,"num_objects":48,"num_object_clones":14,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":20,"num_read":527,"num_read_kb":6360,"num_write":202,"num_write_kb":8951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.176035+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:25:50.176035+0000","last_peered":"2024-05-08T08:25:50.176035+0000","last_clean":"2024-05-08T08:25:50.176035+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:25:50.176035+0000","last_undegraded":"2024-05-08T08:25:50.176035+0000","last_fullsized":"2024-05-08T08:25:50.176035+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:54.867771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"52'247","reported_seq":1489,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.151756+0000","last_change":"2024-05-08T08:25:47.151756+0000","last_active":"2024-05-08T08:25:45.225592+0000","last_peered":"2024-05-08T08:25:43.120530+0000","last_clean":"2024-05-08T08:25:18.938588+0000","last_became_active":"2024-05-08T08:25:21.002993+0000","last_became_peered":"2024-05-08T08:25:21.002993+0000","last_unstale":"2024-05-08T08:25:47.151756+0000","last_undegraded":"2024-05-08T08:25:47.151756+0000","last_fullsized":"2024-05-08T08:25:47.151756+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":247,"log_dups_size":0,"ondisk_log_size":247,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847699999999999,"stat_sum":{"num_bytes":36763536,"num_objects":36,"num_object_clones":10,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":2124,"num_read_kb":25312,"num_write":253,"num_write_kb":4436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174067+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:25:50.174067+0000","last_peered":"2024-05-08T08:25:50.174067+0000","last_clean":"2024-05-08T08:25:50.174067+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:25:50.174067+0000","last_undegraded":"2024-05-08T08:25:50.174067+0000","last_fullsized":"2024-05-08T08:25:50.174067+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:43:18.265730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"60'597","reported_seq":2813,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.316168+0000","last_change":"2024-05-08T08:25:06.611208+0000","last_active":"2024-05-08T08:25:50.316168+0000","last_peered":"2024-05-08T08:25:50.316168+0000","last_clean":"2024-05-08T08:25:50.316168+0000","last_became_active":"2024-05-08T08:25:04.608555+0000","last_became_peered":"2024-05-08T08:25:04.608555+0000","last_unstale":"2024-05-08T08:25:50.316168+0000","last_undegraded":"2024-05-08T08:25:50.316168+0000","last_fullsized":"2024-05-08T08:25:50.316168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:04.768564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195331,"stat_sum":{"num_bytes":44612987,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":17,"num_read":5039,"num_read_kb":59082,"num_write":858,"num_write_kb":32480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.206174+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:25:50.206174+0000","last_peered":"2024-05-08T08:25:50.206174+0000","last_clean":"2024-05-08T08:25:50.206174+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:25:50.206174+0000","last_undegraded":"2024-05-08T08:25:50.206174+0000","last_fullsized":"2024-05-08T08:25:50.206174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"59'237","reported_seq":1720,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.318487+0000","last_change":"2024-05-08T08:25:48.159785+0000","last_active":"2024-05-08T08:25:50.318487+0000","last_peered":"2024-05-08T08:25:50.318487+0000","last_clean":"2024-05-08T08:25:50.318487+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:25:50.318487+0000","last_undegraded":"2024-05-08T08:25:50.318487+0000","last_fullsized":"2024-05-08T08:25:50.318487+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":237,"log_dups_size":0,"ondisk_log_size":237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:53:07.930167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062277100000000005,"stat_sum":{"num_bytes":35575901,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":2222,"num_read_kb":35423,"num_write":257,"num_write_kb":16765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"59'378","reported_seq":1409,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.171762+0000","last_change":"2024-05-08T08:25:06.628002+0000","last_active":"2024-05-08T08:25:50.171762+0000","last_peered":"2024-05-08T08:25:50.171762+0000","last_clean":"2024-05-08T08:25:50.171762+0000","last_became_active":"2024-05-08T08:25:04.615077+0000","last_became_peered":"2024-05-08T08:25:04.615077+0000","last_unstale":"2024-05-08T08:25:50.171762+0000","last_undegraded":"2024-05-08T08:25:50.171762+0000","last_fullsized":"2024-05-08T08:25:50.171762+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:44:08.908263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057249000000000004,"stat_sum":{"num_bytes":11737820,"num_objects":46,"num_object_clones":12,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":23,"num_read":2433,"num_read_kb":9657,"num_write":442,"num_write_kb":6631,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":255,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238459+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:25:50.238459+0000","last_peered":"2024-05-08T08:25:50.238459+0000","last_clean":"2024-05-08T08:25:50.238459+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:25:50.238459+0000","last_undegraded":"2024-05-08T08:25:50.238459+0000","last_fullsized":"2024-05-08T08:25:50.238459+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":24,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-05-08T08:25:47.151317+0000","last_change":"2024-05-08T08:25:45.225724+0000","last_active":"2024-05-08T08:25:47.151317+0000","last_peered":"2024-05-08T08:25:47.151317+0000","last_clean":"2024-05-08T08:25:47.151317+0000","last_became_active":"2024-05-08T08:25:45.225388+0000","last_became_peered":"2024-05-08T08:25:45.225388+0000","last_unstale":"2024-05-08T08:25:47.151317+0000","last_undegraded":"2024-05-08T08:25:47.151317+0000","last_fullsized":"2024-05-08T08:25:47.151317+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:54.551477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"56'185","reported_seq":883,"reported_epoch":57,"state":"peering","last_fresh":"2024-05-08T08:25:47.154031+0000","last_change":"2024-05-08T08:25:47.151090+0000","last_active":"2024-05-08T08:25:47.151090+0000","last_peered":"2024-05-08T08:25:46.732061+0000","last_clean":"2024-05-08T08:25:18.936967+0000","last_became_active":"2024-05-08T08:25:46.260405+0000","last_became_peered":"2024-05-08T08:25:46.260405+0000","last_unstale":"2024-05-08T08:25:47.154031+0000","last_undegraded":"2024-05-08T08:25:47.154031+0000","last_fullsized":"2024-05-08T08:25:47.154031+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034192400000000001,"stat_sum":{"num_bytes":23073517,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":580,"num_read_kb":8154,"num_write":223,"num_write_kb":8181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238782+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:25:50.238782+0000","last_peered":"2024-05-08T08:25:50.238782+0000","last_clean":"2024-05-08T08:25:50.238782+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:25:50.238782+0000","last_undegraded":"2024-05-08T08:25:50.238782+0000","last_fullsized":"2024-05-08T08:25:50.238782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"60'210","reported_seq":805,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.314514+0000","last_change":"2024-05-08T08:25:06.615595+0000","last_active":"2024-05-08T08:25:50.314514+0000","last_peered":"2024-05-08T08:25:50.314514+0000","last_clean":"2024-05-08T08:25:50.314514+0000","last_became_active":"2024-05-08T08:25:04.611900+0000","last_became_peered":"2024-05-08T08:25:04.611900+0000","last_unstale":"2024-05-08T08:25:50.314514+0000","last_undegraded":"2024-05-08T08:25:50.314514+0000","last_fullsized":"2024-05-08T08:25:50.314514+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":210,"log_dups_size":0,"ondisk_log_size":210,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:15:47.500524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050119499999999996,"stat_sum":{"num_bytes":19631352,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":12,"num_read":533,"num_read_kb":20098,"num_write":219,"num_write_kb":15522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174391+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:25:50.174391+0000","last_peered":"2024-05-08T08:25:50.174391+0000","last_clean":"2024-05-08T08:25:50.174391+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:25:50.174391+0000","last_undegraded":"2024-05-08T08:25:50.174391+0000","last_fullsized":"2024-05-08T08:25:50.174391+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"59'149","reported_seq":1103,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.315072+0000","last_change":"2024-05-08T08:25:06.610137+0000","last_active":"2024-05-08T08:25:50.315072+0000","last_peered":"2024-05-08T08:25:50.315072+0000","last_clean":"2024-05-08T08:25:50.315072+0000","last_became_active":"2024-05-08T08:25:04.606801+0000","last_became_peered":"2024-05-08T08:25:04.606801+0000","last_unstale":"2024-05-08T08:25:50.315072+0000","last_undegraded":"2024-05-08T08:25:50.315072+0000","last_fullsized":"2024-05-08T08:25:50.315072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:04.848361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042592699999999999,"stat_sum":{"num_bytes":61920517,"num_objects":38,"num_object_clones":10,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":19,"num_read":899,"num_read_kb":46926,"num_write":151,"num_write_kb":20177,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238584+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:25:50.238584+0000","last_peered":"2024-05-08T08:25:50.238584+0000","last_clean":"2024-05-08T08:25:50.238584+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:25:50.238584+0000","last_undegraded":"2024-05-08T08:25:50.238584+0000","last_fullsized":"2024-05-08T08:25:50.238584+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"56'216","reported_seq":2169,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.206318+0000","last_change":"2024-05-08T08:25:48.159093+0000","last_active":"2024-05-08T08:25:50.206318+0000","last_peered":"2024-05-08T08:25:50.206318+0000","last_clean":"2024-05-08T08:25:50.206318+0000","last_became_active":"2024-05-08T08:25:48.158198+0000","last_became_peered":"2024-05-08T08:25:48.158198+0000","last_unstale":"2024-05-08T08:25:50.206318+0000","last_undegraded":"2024-05-08T08:25:50.206318+0000","last_fullsized":"2024-05-08T08:25:50.206318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":216,"log_dups_size":0,"ondisk_log_size":216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.689664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001097338,"stat_sum":{"num_bytes":23653360,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":15,"num_read":3084,"num_read_kb":21242,"num_write":250,"num_write_kb":16653,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"59'218","reported_seq":1078,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174056+0000","last_change":"2024-05-08T08:25:06.615379+0000","last_active":"2024-05-08T08:25:50.174056+0000","last_peered":"2024-05-08T08:25:50.174056+0000","last_clean":"2024-05-08T08:25:50.174056+0000","last_became_active":"2024-05-08T08:25:04.901435+0000","last_became_peered":"2024-05-08T08:25:04.901435+0000","last_unstale":"2024-05-08T08:25:50.174056+0000","last_undegraded":"2024-05-08T08:25:50.174056+0000","last_fullsized":"2024-05-08T08:25:50.174056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":218,"log_dups_size":0,"ondisk_log_size":218,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:22:02.472778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039550799999999998,"stat_sum":{"num_bytes":11085840,"num_objects":29,"num_object_clones":8,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":15,"num_read":2089,"num_read_kb":6401,"num_write":209,"num_write_kb":3268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"60'366","reported_seq":824,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.313079+0000","last_change":"2024-05-08T08:25:06.612336+0000","last_active":"2024-05-08T08:25:50.313079+0000","last_peered":"2024-05-08T08:25:50.313079+0000","last_clean":"2024-05-08T08:25:50.313079+0000","last_became_active":"2024-05-08T08:25:04.615227+0000","last_became_peered":"2024-05-08T08:25:04.615227+0000","last_unstale":"2024-05-08T08:25:50.313079+0000","last_undegraded":"2024-05-08T08:25:50.313079+0000","last_fullsized":"2024-05-08T08:25:50.313079+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":366,"log_dups_size":0,"ondisk_log_size":366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:23.233411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034968800000000002,"stat_sum":{"num_bytes":28282536,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":1467,"num_read_kb":16390,"num_write":505,"num_write_kb":7440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"58'176","reported_seq":1328,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.312050+0000","last_change":"2024-05-08T08:25:06.610868+0000","last_active":"2024-05-08T08:25:50.312050+0000","last_peered":"2024-05-08T08:25:50.312050+0000","last_clean":"2024-05-08T08:25:50.312050+0000","last_became_active":"2024-05-08T08:25:04.608456+0000","last_became_peered":"2024-05-08T08:25:04.608456+0000","last_unstale":"2024-05-08T08:25:50.312050+0000","last_undegraded":"2024-05-08T08:25:50.312050+0000","last_fullsized":"2024-05-08T08:25:50.312050+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:45:37.416621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112149,"stat_sum":{"num_bytes":34120901,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":17,"num_read":1096,"num_read_kb":14290,"num_write":176,"num_write_kb":8770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"59'192","reported_seq":864,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.174647+0000","last_change":"2024-05-08T08:25:06.624428+0000","last_active":"2024-05-08T08:25:50.174647+0000","last_peered":"2024-05-08T08:25:50.174647+0000","last_clean":"2024-05-08T08:25:50.174647+0000","last_became_active":"2024-05-08T08:25:04.606925+0000","last_became_peered":"2024-05-08T08:25:04.606925+0000","last_unstale":"2024-05-08T08:25:50.174647+0000","last_undegraded":"2024-05-08T08:25:50.174647+0000","last_fullsized":"2024-05-08T08:25:50.174647+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":192,"log_dups_size":0,"ondisk_log_size":192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:49:24.692160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046376499999999999,"stat_sum":{"num_bytes":58995571,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":614,"num_read_kb":34697,"num_write":264,"num_write_kb":22805,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"59'177","reported_seq":468,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.315221+0000","last_change":"2024-05-08T08:25:06.796419+0000","last_active":"2024-05-08T08:25:50.315221+0000","last_peered":"2024-05-08T08:25:50.315221+0000","last_clean":"2024-05-08T08:25:50.315221+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:25:50.315221+0000","last_undegraded":"2024-05-08T08:25:50.315221+0000","last_fullsized":"2024-05-08T08:25:50.315221+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":177,"log_dups_size":0,"ondisk_log_size":177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:45.240770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046517299999999999,"stat_sum":{"num_bytes":10995804,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":216,"num_read_kb":7926,"num_write":178,"num_write_kb":4861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"59'322","reported_seq":2341,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.238601+0000","last_change":"2024-05-08T08:25:06.796635+0000","last_active":"2024-05-08T08:25:50.238601+0000","last_peered":"2024-05-08T08:25:50.238601+0000","last_clean":"2024-05-08T08:25:50.238601+0000","last_became_active":"2024-05-08T08:25:04.615004+0000","last_became_peered":"2024-05-08T08:25:04.615004+0000","last_unstale":"2024-05-08T08:25:50.238601+0000","last_undegraded":"2024-05-08T08:25:50.238601+0000","last_fullsized":"2024-05-08T08:25:50.238601+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":322,"log_dups_size":0,"ondisk_log_size":322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:04.429594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060116800000000001,"stat_sum":{"num_bytes":43995266,"num_objects":25,"num_object_clones":3,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":13,"num_read":3394,"num_read_kb":37060,"num_write":333,"num_write_kb":22695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"59'213","reported_seq":728,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239137+0000","last_change":"2024-05-08T08:25:06.796737+0000","last_active":"2024-05-08T08:25:50.239137+0000","last_peered":"2024-05-08T08:25:50.239137+0000","last_clean":"2024-05-08T08:25:50.239137+0000","last_became_active":"2024-05-08T08:25:04.615942+0000","last_became_peered":"2024-05-08T08:25:04.615942+0000","last_unstale":"2024-05-08T08:25:50.239137+0000","last_undegraded":"2024-05-08T08:25:50.239137+0000","last_fullsized":"2024-05-08T08:25:50.239137+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":213,"log_dups_size":0,"ondisk_log_size":213,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:54.886102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065917600000000003,"stat_sum":{"num_bytes":5724433,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":10,"num_read":794,"num_read_kb":2629,"num_write":252,"num_write_kb":1884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"60'546","reported_seq":1135,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.278796+0000","last_change":"2024-05-08T08:25:06.612297+0000","last_active":"2024-05-08T08:25:50.278796+0000","last_peered":"2024-05-08T08:25:50.278796+0000","last_clean":"2024-05-08T08:25:50.278796+0000","last_became_active":"2024-05-08T08:25:04.616212+0000","last_became_peered":"2024-05-08T08:25:04.616212+0000","last_unstale":"2024-05-08T08:25:50.278796+0000","last_undegraded":"2024-05-08T08:25:50.278796+0000","last_fullsized":"2024-05-08T08:25:50.278796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":546,"log_dups_size":0,"ondisk_log_size":546,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:29:31.116080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031704699999999999,"stat_sum":{"num_bytes":5284092,"num_objects":32,"num_object_clones":8,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":10,"num_read":2537,"num_read_kb":6619,"num_write":839,"num_write_kb":3465,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"59'204","reported_seq":933,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.239287+0000","last_change":"2024-05-08T08:25:06.796231+0000","last_active":"2024-05-08T08:25:50.239287+0000","last_peered":"2024-05-08T08:25:50.239287+0000","last_clean":"2024-05-08T08:25:50.239287+0000","last_became_active":"2024-05-08T08:25:04.615031+0000","last_became_peered":"2024-05-08T08:25:04.615031+0000","last_unstale":"2024-05-08T08:25:50.239287+0000","last_undegraded":"2024-05-08T08:25:50.239287+0000","last_fullsized":"2024-05-08T08:25:50.239287+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":204,"log_dups_size":0,"ondisk_log_size":204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:30:30.412714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040064000000000001,"stat_sum":{"num_bytes":23621913,"num_objects":33,"num_object_clones":9,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":14,"num_read":829,"num_read_kb":19159,"num_write":222,"num_write_kb":5194,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"58'253","reported_seq":1153,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.316670+0000","last_change":"2024-05-08T08:25:06.628036+0000","last_active":"2024-05-08T08:25:50.316670+0000","last_peered":"2024-05-08T08:25:50.316670+0000","last_clean":"2024-05-08T08:25:50.316670+0000","last_became_active":"2024-05-08T08:25:04.609651+0000","last_became_peered":"2024-05-08T08:25:04.609651+0000","last_unstale":"2024-05-08T08:25:50.316670+0000","last_undegraded":"2024-05-08T08:25:50.316670+0000","last_fullsized":"2024-05-08T08:25:50.316670+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":253,"log_dups_size":0,"ondisk_log_size":253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:47:30.467433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050895200000000002,"stat_sum":{"num_bytes":20883517,"num_objects":38,"num_object_clones":9,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":13,"num_read":1350,"num_read_kb":20411,"num_write":319,"num_write_kb":13515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"60'514","reported_seq":1361,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-05-08T08:25:50.240943+0000","last_change":"2024-05-08T08:25:06.796497+0000","last_active":"2024-05-08T08:25:50.240943+0000","last_peered":"2024-05-08T08:25:50.240943+0000","last_clean":"2024-05-08T08:25:50.240943+0000","last_became_active":"2024-05-08T08:25:04.616040+0000","last_became_peered":"2024-05-08T08:25:04.616040+0000","last_unstale":"2024-05-08T08:25:50.240943+0000","last_undegraded":"2024-05-08T08:25:50.240943+0000","last_fullsized":"2024-05-08T08:25:50.240943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:51.767601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050921500000000004,"stat_sum":{"num_bytes":48287375,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":2748,"num_read_kb":15447,"num_write":830,"num_write_kb":12206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":894868603,"num_objects":1122,"num_object_clones":261,"num_object_copies":2244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1122,"num_whiteouts":462,"num_read":60162,"num_read_kb":610107,"num_write":12072,"num_write_kb":350973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":209,"num_bytes_recovered":13745240,"num_keys_recovered":12,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45748224,"data_stored":518901662,"data_compressed":16329872,"data_compressed_allocated":39673856,"data_compressed_original":515224942,"omap_allocated":9107,"internal_metadata":0},"log_size":9435,"ondisk_log_size":9435,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47032,"kb_used_data":7528,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94324808,"statfs":{"total":96636764160,"available":96588603392,"internally_reserved":0,"allocated":7708672,"data_stored":85619922,"data_compressed":2756454,"data_compressed_allocated":6094848,"data_compressed_original":84713283,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33688,"kb_used_data":2492,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338152,"statfs":{"total":96636764160,"available":96602267648,"internally_reserved":0,"allocated":2551808,"data_stored":21471733,"data_compressed":615042,"data_compressed_allocated":1548288,"data_compressed_original":21004288,"omap_allocated":1592,"internal_metadata":31914440},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43352,"kb_used_data":8016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94328488,"statfs":{"total":96636764160,"available":96592371712,"internally_reserved":0,"allocated":8208384,"data_stored":85986856,"data_compressed":2559614,"data_compressed_allocated":6623232,"data_compressed_original":85109415,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53140,"kb_used_data":8072,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94318700,"statfs":{"total":96636764160,"available":96582348800,"internally_reserved":0,"allocated":8265728,"data_stored":85814967,"data_compressed":2840977,"data_compressed_allocated":6426624,"data_compressed_original":84793846,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574863,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45168,"kb_used_data":9816,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94326672,"statfs":{"total":96636764160,"available":96590512128,"internally_reserved":0,"allocated":10051584,"data_stored":103736403,"data_compressed":3214260,"data_compressed_allocated":8335360,"data_compressed_original":102821368,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574862,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33500,"kb_used_data":2320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338340,"statfs":{"total":96636764160,"available":96602460160,"internally_reserved":0,"allocated":2375680,"data_stored":16593268,"data_compressed":721391,"data_compressed_allocated":1368064,"data_compressed_original":16076703,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":54,"seq":231928233986,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33676,"kb_used_data":4992,"kb_used_omap":13,"kb_used_meta":28658,"kb_avail":94338164,"statfs":{"total":96636764160,"available":96602279936,"internally_reserved":0,"allocated":5111808,"data_stored":46096708,"data_compressed":1232659,"data_compressed_allocated":3903488,"data_compressed_original":45473792,"omap_allocated":13550,"internal_metadata":29346578},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":68588,"kb_used_data":6884,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94303252,"statfs":{"total":96636764160,"available":96566530048,"internally_reserved":0,"allocated":7049216,"data_stored":77230786,"data_compressed":2399361,"data_compressed_allocated":5447680,"data_compressed_original":76379127,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6369280,"data_stored":76922333,"data_compressed":2399361,"data_compressed_allocated":5447680,"data_compressed_original":76379127,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4395008,"data_stored":45209515,"data_compressed":1227716,"data_compressed_allocated":3866624,"data_compressed_original":44900352,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1679360,"data_stored":16278634,"data_compressed":721391,"data_compressed_allocated":1368064,"data_compressed_original":16076703,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9375744,"data_stored":103428184,"data_compressed":3214260,"data_compressed_allocated":8335360,"data_compressed_original":102821368,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7581696,"data_stored":85506484,"data_compressed":2840977,"data_compressed_allocated":6426624,"data_compressed_original":84793846,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7524352,"data_stored":85678342,"data_compressed":2559614,"data_compressed_allocated":6623232,"data_compressed_original":85109415,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1855488,"data_stored":21157099,"data_compressed":615042,"data_compressed_allocated":1548288,"data_compressed_original":21004288,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6967296,"data_stored":84721071,"data_compressed":2751511,"data_compressed_allocated":6057984,"data_compressed_original":84139843,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:53.056 INFO:teuthology.orchestra.run.smithi031.stdout:3998 read 0x6495aa thru 0x655925 (0xc37c bytes) 2024-05-08T08:25:53.056 INFO:teuthology.orchestra.run.smithi031.stdout:3999 writesame 0x401ade3 thru 0x40299cb data_size 0x29(0xebe9 bytes) 2024-05-08T08:25:53.056 INFO:teuthology.orchestra.run.smithi031.stdout:4000 write 0x7fd5c1 thru 0x801e46 (0x4886 bytes) 2024-05-08T08:25:53.056 INFO:teuthology.orchestra.run.smithi031.stdout:4003 write 0x9f64f9e thru 0x9f68421 (0x3484 bytes) 2024-05-08T08:25:53.057 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:53.057+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:53.065 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.065 INFO:teuthology.orchestra.run.smithi031.stdout:4006 punch from 0x97325a to 0x981cdd, (0xea83 bytes) 2024-05-08T08:25:53.065 INFO:teuthology.orchestra.run.smithi031.stdout:4007 read 0x3b9285 thru 0x3c3927 (0xa6a3 bytes) 2024-05-08T08:25:53.070 INFO:teuthology.orchestra.run.smithi031.stdout:4008 trunc from 0x9f68422 to 0xd26edd2 2024-05-08T08:25:53.075 INFO:teuthology.orchestra.run.smithi031.stdout:4009 read 0x7011425 thru 0x701fe89 (0xea65 bytes) 2024-05-08T08:25:53.075 INFO:teuthology.orchestra.run.smithi031.stdout:4010 read 0xbf0f43d thru 0xbf1357e (0x4142 bytes) 2024-05-08T08:25:53.075 INFO:teuthology.orchestra.run.smithi031.stdout:4011 write 0x41721d3 thru 0x4181724 (0xf552 bytes) 2024-05-08T08:25:53.087 INFO:teuthology.orchestra.run.smithi031.stdout:4014 write 0xa216a6 thru 0xa2fec0 (0xe81b bytes) 2024-05-08T08:25:53.091 INFO:teuthology.orchestra.run.smithi031.stdout:4015 writesame 0x42e1848 thru 0x42ececc data_size 0x69(0xb685 bytes) 2024-05-08T08:25:53.097 INFO:teuthology.orchestra.run.smithi031.stdout:4016 read 0x1323f9d thru 0x1325daa (0x1e0e bytes) 2024-05-08T08:25:53.097 INFO:teuthology.orchestra.run.smithi031.stdout:4017 write 0xb128695 thru 0xb12cda6 (0x4712 bytes) 2024-05-08T08:25:53.109 INFO:teuthology.orchestra.run.smithi031.stdout:4018 read 0x762d67b thru 0x7632e78 (0x57fe bytes) 2024-05-08T08:25:53.109 INFO:teuthology.orchestra.run.smithi031.stdout:4019 clone 25 order 18 su 32768 sc 8 2024-05-08T08:25:53.151 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:53.158+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:53.184 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:53.184 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:53.184 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:53.184 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:53.194 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:53.252 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:53.260+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:53.350 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:53.351 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:53.351 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:53.351 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:53.351 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:53.353 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:53.360+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:53.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:53.377 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-05-08T08:25:53.378 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.379 INFO:teuthology.orchestra.run.smithi031.stdout:4023 write 0x8110a3b thru 0x81149c1 (0x3f87 bytes) 2024-05-08T08:25:53.384 INFO:teuthology.orchestra.run.smithi031.stdout:4026 write 0x93dc7cd thru 0x93eb85e (0xf092 bytes) 2024-05-08T08:25:53.389 INFO:teuthology.orchestra.run.smithi031.stdout:4028 write 0x839c2b7 thru 0x839e9ee (0x2738 bytes) 2024-05-08T08:25:53.392 INFO:teuthology.orchestra.run.smithi031.stdout:4029 punch from 0x563c093 to 0x564aabf, (0xea2c bytes) 2024-05-08T08:25:53.392 INFO:teuthology.orchestra.run.smithi031.stdout:4032 write 0x1cb086e thru 0x1cbcf13 (0xc6a6 bytes) 2024-05-08T08:25:53.399 INFO:teuthology.orchestra.run.smithi031.stdout:4033 write 0x1fbea80 thru 0x1fcb0c6 (0xc647 bytes) 2024-05-08T08:25:53.407 INFO:teuthology.orchestra.run.smithi031.stdout:4034 write 0x476169f thru 0x476516f (0x3ad1 bytes) 2024-05-08T08:25:53.410 INFO:teuthology.orchestra.run.smithi031.stdout:4038 punch from 0x7b5e646 to 0x7b6d1ba, (0xeb74 bytes) 2024-05-08T08:25:53.411 INFO:teuthology.orchestra.run.smithi031.stdout:4040 read 0x638f75c thru 0x639912b (0x99d0 bytes) 2024-05-08T08:25:53.411 INFO:teuthology.orchestra.run.smithi031.stdout:4042 write 0x10e40f5 thru 0x10e798a (0x3896 bytes) 2024-05-08T08:25:53.415 INFO:teuthology.orchestra.run.smithi031.stdout:4043 read 0x106b11f thru 0x107064d (0x552f bytes) 2024-05-08T08:25:53.454 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:53.462+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:53.458 INFO:teuthology.orchestra.run.smithi031.stdout:4044 read 0x4cfa312 thru 0x4d0065d (0x634c bytes) 2024-05-08T08:25:53.458 INFO:teuthology.orchestra.run.smithi031.stdout:4045 write 0x17618b2 thru 0x1762f21 (0x1670 bytes) 2024-05-08T08:25:53.466 INFO:teuthology.orchestra.run.smithi031.stdout:4046 read 0x4e2055c thru 0x4e20a68 (0x50d bytes) 2024-05-08T08:25:53.466 INFO:teuthology.orchestra.run.smithi031.stdout:4047 write 0x71aa520 thru 0x71afb54 (0x5635 bytes) 2024-05-08T08:25:53.470 INFO:teuthology.orchestra.run.smithi031.stdout:4048 punch from 0xb25d6d8 to 0xb26bea0, (0xe7c8 bytes) 2024-05-08T08:25:53.470 INFO:teuthology.orchestra.run.smithi031.stdout:4049 read 0x263dc91 thru 0x26489a6 (0xad16 bytes) 2024-05-08T08:25:53.470 INFO:teuthology.orchestra.run.smithi031.stdout:4050 read 0xc3db688 thru 0xc3dea94 (0x340d bytes) 2024-05-08T08:25:53.470 INFO:teuthology.orchestra.run.smithi031.stdout:4051 punch from 0x8c15fe8 to 0x8c18c27, (0x2c3f bytes) 2024-05-08T08:25:53.470 INFO:teuthology.orchestra.run.smithi031.stdout:4052 write 0x5d6bd75 thru 0x5d735ab (0x7837 bytes) 2024-05-08T08:25:53.474 INFO:teuthology.orchestra.run.smithi031.stdout:4053 punch from 0x35d8c53 to 0x35e3f04, (0xb2b1 bytes) 2024-05-08T08:25:53.496 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6562 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.571622810999997, 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024048491000000002, 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6562 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:53.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931270+0000", 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.19e-06 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931276+0000", 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4790000000000004e-06 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931286+0000", 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.044e-05 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931292+0000", 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931346+0000", 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4274999999999998e-05 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931398+0000", 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1999999999999997e-05 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.935504+0000", 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041055170000000004 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955221+0000", 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019717443000000001 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955234+0000", 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2191999999999999e-05 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955271+0000", 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7008000000000002e-05 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955317+0000", 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6796999999999999e-05 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7004 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7077888~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314672963, 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016625803000000001, 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.502 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7004 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188222+0000", 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7219999999999999e-06 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188229+0000", 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6359999999999997e-06 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188234+0000", 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.502e-06 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203959+0000", 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015725330999999999 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204038+0000", 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8379000000000003e-05 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204845+0000", 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080675500000000004 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.506 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7020 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314583763999998, 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017204636999999998, 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7020 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188311+0000", 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188317+0000", 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7959999999999998e-06 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188322+0000", 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204639+0000", 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016316714 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204730+0000", 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0771000000000002e-05 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205513+0000", 2024-05-08T08:25:53.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078277700000000002 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7021 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3145728~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314562787, 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017850441000000002, 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.511 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7021 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6199999999999999e-07 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188331+0000", 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1069999999999999e-06 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188338+0000", 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.513 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188343+0000", 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205565+0000", 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017221841000000002 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205665+0000", 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9530000000000004e-05 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206179+0000", 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051477999999999995 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7022 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 5767168~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314541667, 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018456931999999999, 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.515 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7022 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188352+0000", 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188358+0000", 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188363+0000", 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206229+0000", 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017865941 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206327+0000", 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7888999999999997e-05 2024-05-08T08:25:53.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206807+0000", 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048019000000000002 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7038 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314435762999999, 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019193815999999999, 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.519 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7038 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188457+0000", 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188461+0000", 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188469+0000", 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188477+0000", 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3559999999999993e-06 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206829+0000", 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018352456 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206928+0000", 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8981999999999995e-05 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207650+0000", 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072148600000000002 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7039 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314403022, 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491153000000001, 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.523 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7039 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188492+0000", 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188499+0000", 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7469999999999999e-06 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188505+0000", 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207495+0000", 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990683000000001 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207577+0000", 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.174e-05 2024-05-08T08:25:53.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207980+0000", 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040296899999999998 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7040 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314379733000003, 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020064733000000001, 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.527 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7040 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188516+0000", 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999999e-06 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188522+0000", 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188527+0000", 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208019+0000", 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491871000000001 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208118+0000", 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8855000000000001e-05 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208577+0000", 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000458784 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7056 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314298909999998, 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020583127999999999, 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.531 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188595+0000", 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0679999999999999e-06 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188603+0000", 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999997e-06 2024-05-08T08:25:53.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188609+0000", 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208623+0000", 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0200141 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208713+0000", 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9806e-05 2024-05-08T08:25:53.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209176+0000", 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046324300000000001 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7057 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4718592~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314245792999998, 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021149303000000001, 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188647+0000", 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188650+0000", 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7100000000000001e-06 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188657+0000", 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-05-08T08:25:53.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188665+0000", 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4410000000000004e-06 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209211+0000", 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020546426999999999 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209299+0000", 2024-05-08T08:25:53.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8139000000000004e-05 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209795+0000", 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049584800000000001 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7058 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7340032~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314215480000001, 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021706842, 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7058 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:53.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188677+0000", 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188681+0000", 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188688+0000", 2024-05-08T08:25:53.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.494e-06 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188693+0000", 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209838+0000", 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021144677000000001 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209901+0000", 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3322000000000001e-05 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210383+0000", 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048203800000000001 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7073 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2359296~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314119970999997, 2024-05-08T08:25:53.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022130316000000001, 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188775+0000", 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188782+0000", 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8630000000000008e-06 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188787+0000", 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210421+0000", 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021633454999999999 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210481+0000", 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9911000000000003e-05 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210902+0000", 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042136000000000002 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7074 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4980736~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314097674999999, 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022761245999999999, 2024-05-08T08:25:53.547 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188795+0000", 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188797+0000", 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188803+0000", 2024-05-08T08:25:53.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188807+0000", 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210951+0000", 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022143843 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211043+0000", 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2044000000000002e-05 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211555+0000", 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051209900000000002 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7075 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7602176~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:53.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.314077748000003, 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025123269, 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7075 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188817+0000", 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188823+0000", 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188828+0000", 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211585+0000", 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022757553999999999 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211640+0000", 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000002e-05 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213937+0000", 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002297182 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7090 3.12 3:4c3c5acc:::rbd_data.119120fdbc10.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1572864~19660800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:53.555 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.304603624000002, 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017096892999999998, 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7090 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198290+0000", 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.666e-06 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198297+0000", 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198308+0000", 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0641e-05 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198314+0000", 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7850000000000003e-06 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213971+0000", 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015656415 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214141+0000", 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016982999999999999 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214349+0000", 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020874199999999999 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215032+0000", 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068226599999999995 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215223+0000", 2024-05-08T08:25:53.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191263 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215286+0000", 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000004e-05 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215385+0000", 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9161999999999997e-05 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.560 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10928 3.6 3:66234790:::rbd_data.11917ede465c.0000000000000000:9 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.363893847, 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016562112, 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10928 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:53.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139012+0000", 2024-05-08T08:25:53.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3908e-05 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139020+0000", 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.96e-06 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139026+0000", 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5219999999999998e-06 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155334+0000", 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016308699999999999 2024-05-08T08:25:53.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155417+0000", 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2229000000000002e-05 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155560+0000", 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014340300000000001 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10943 3.6 3:62dc9057:::rbd_data.119177d1b31a.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:53.564 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.362131861000002, 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018993326000000001, 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10943 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140761+0000", 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999999e-07 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140765+0000", 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140775+0000", 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9690000000000006e-06 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140782+0000", 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156122+0000", 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015340006999999999 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156209+0000", 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7249000000000004e-05 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159642+0000", 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034333430000000002 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159700+0000", 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8078999999999997e-05 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159753+0000", 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3019000000000002e-05 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:13022 3.17 3:ee16c3de:::rbd_data.1191c87c6f01.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 20828828~4380244] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.968957404000001, 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020689712999999998, 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13022 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533938+0000", 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9879999999999998e-06 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534933+0000", 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099514400000000011 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534955+0000", 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1225000000000001e-05 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534965+0000", 2024-05-08T08:25:53.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597999999999999e-05 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535030+0000", 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5068999999999997e-05 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535157+0000", 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126501 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535401+0000", 2024-05-08T08:25:53.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024435700000000001 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.540584+0000", 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051824109999999996 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554524+0000", 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013939890999999999 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554584+0000", 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:53.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554624+0000", 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0028999999999999e-05 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:23289 3.3 e41/18 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head v 41'115, mlcod=39'113)", 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.979202624999999, 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024290019999999999, 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.574 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523690+0000", 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2900000000000002e-07 2024-05-08T08:25:53.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545794+0000", 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022104220000000001 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545810+0000", 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5420000000000001e-05 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545818+0000", 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0979999999999994e-06 2024-05-08T08:25:53.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545894+0000", 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6579e-05 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545914+0000", 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9372e-05 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547942+0000", 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020284220000000002 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547979+0000", 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6980000000000002e-05 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:25389 3.17 3:ea3e87a1:::rbd_data.11911e381319.0000000000000114:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.55489875, 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020343165999999999, 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:53.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 25389 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947994+0000", 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1319999999999999e-06 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947998+0000", 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2830000000000003e-06 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948009+0000", 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.058e-05 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948014+0000", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948057+0000", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2330000000000003e-05 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948137+0000", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9985000000000001e-05 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.952162+0000", 2024-05-08T08:25:53.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040247959999999998 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968241+0000", 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016079751 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968253+0000", 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1296e-05 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968290+0000", 2024-05-08T08:25:53.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7011000000000002e-05 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968336+0000", 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6612e-05 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:53.583 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:53.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:53.585 INFO:teuthology.orchestra.run.smithi031.stdout:4054 punch from 0x92cd9b7 to 0x92ce161, (0x7aa bytes) 2024-05-08T08:25:53.585 INFO:teuthology.orchestra.run.smithi031.stdout:4055 read 0x47e2ff9 thru 0x47edafa (0xab02 bytes) 2024-05-08T08:25:53.585 INFO:teuthology.orchestra.run.smithi031.stdout:4057 read 0x9fcc204 thru 0x9fd536b (0x9168 bytes) 2024-05-08T08:25:53.585 INFO:teuthology.orchestra.run.smithi031.stdout:4058 trunc from 0xd26edd2 to 0x830f254 2024-05-08T08:25:53.586 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:53.592+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:53.587 INFO:teuthology.orchestra.run.smithi031.stdout:4059 trunc from 0x830f254 to 0x6d9b3d2 2024-05-08T08:25:53.615 INFO:teuthology.orchestra.run.smithi031.stdout:4060 read 0x3b82109 thru 0x3b8c260 (0xa158 bytes) 2024-05-08T08:25:53.615 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.615 INFO:teuthology.orchestra.run.smithi031.stdout:4063 write 0x5c532b9 thru 0x5c61bbe (0xe906 bytes) 2024-05-08T08:25:53.625 INFO:teuthology.orchestra.run.smithi031.stdout:4064 punch from 0x631d751 to 0x631fe6d, (0x271c bytes) 2024-05-08T08:25:53.632 INFO:teuthology.orchestra.run.smithi031.stdout:4065 write 0x7f58cba thru 0x7f67ef0 (0xf237 bytes) 2024-05-08T08:25:53.639 INFO:teuthology.orchestra.run.smithi031.stdout:4066 writesame 0x37686cb thru 0x376e25e data_size 0x2(0x5b94 bytes) 2024-05-08T08:25:53.649 INFO:teuthology.orchestra.run.smithi031.stdout:4067 punch from 0x592dfe7 to 0x5930728, (0x2741 bytes) 2024-05-08T08:25:53.649 INFO:teuthology.orchestra.run.smithi031.stdout:4068 read 0x467dddb thru 0x46805e9 (0x280f bytes) 2024-05-08T08:25:53.659 INFO:teuthology.orchestra.run.smithi031.stdout:4069 trunc from 0x7f67ef1 to 0x20808f 2024-05-08T08:25:53.685 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:53.692+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:53.689 INFO:teuthology.orchestra.run.smithi031.stdout:4070 punch from 0x46dc to 0x5396, (0xcba bytes) 2024-05-08T08:25:53.699 INFO:teuthology.orchestra.run.smithi031.stdout:4071 trunc from 0x20808f to 0x7951e96 2024-05-08T08:25:53.704 INFO:teuthology.orchestra.run.smithi031.stdout:4074 punch from 0x225a197 to 0x2265883, (0xb6ec bytes) 2024-05-08T08:25:53.704 INFO:teuthology.orchestra.run.smithi031.stdout:4075 punch from 0x18adcc7 to 0x18b5e1a, (0x8153 bytes) 2024-05-08T08:25:53.704 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.704 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.711 INFO:teuthology.orchestra.run.smithi031.stdout:4078 trunc from 0x7951e96 to 0xd435995 2024-05-08T08:25:53.716 INFO:teuthology.orchestra.run.smithi031.stdout:4079 write 0x7c25e9e thru 0x7c260ac (0x20f bytes) 2024-05-08T08:25:53.719 INFO:teuthology.orchestra.run.smithi031.stdout:4080 read 0xcfea95f thru 0xcfec6f8 (0x1d9a bytes) 2024-05-08T08:25:53.719 INFO:teuthology.orchestra.run.smithi031.stdout:4081 read 0xd1d15e6 thru 0xd1d77d8 (0x61f3 bytes) 2024-05-08T08:25:53.720 INFO:teuthology.orchestra.run.smithi031.stdout:4084 writesame 0x91038d2 thru 0x910c871 data_size 0x30(0x8fa0 bytes) 2024-05-08T08:25:53.723 INFO:teuthology.orchestra.run.smithi031.stdout:4086 punch from 0x96e41fa to 0x96ec170, (0x7f76 bytes) 2024-05-08T08:25:53.723 INFO:teuthology.orchestra.run.smithi031.stdout:4087 writesame 0x89feec8 thru 0x8a0ca21 data_size 0xe(0xdb5a bytes) 2024-05-08T08:25:53.725 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:53.726 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:53.726 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:53.726 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:53.730 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.731 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.731 INFO:teuthology.orchestra.run.smithi031.stdout:4090 read 0x86b6464 thru 0x86bce3e (0x69db bytes) 2024-05-08T08:25:53.731 INFO:teuthology.orchestra.run.smithi031.stdout:4092 writesame 0x7c3b16d thru 0x7c4b0c3 data_size 0x51(0xff57 bytes) 2024-05-08T08:25:53.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:53.741 INFO:teuthology.orchestra.run.smithi031.stdout:4094 punch from 0x11ae781 to 0x11b1006, (0x2885 bytes) 2024-05-08T08:25:53.741 INFO:teuthology.orchestra.run.smithi031.stdout:4095 writesame 0xc7a3b44 thru 0xc7b28be data_size 0x2d(0xed7b bytes) 2024-05-08T08:25:53.751 INFO:teuthology.orchestra.run.smithi031.stdout:4097 punch from 0x6ada2ad to 0x6ae067a, (0x63cd bytes) 2024-05-08T08:25:53.751 INFO:teuthology.orchestra.run.smithi031.stdout:4099 read 0x3011a3b thru 0x30125e7 (0xbad bytes) 2024-05-08T08:25:53.751 INFO:teuthology.orchestra.run.smithi031.stdout:4100 write 0xc302ae9 thru 0xc30b0b6 (0x85ce bytes) 2024-05-08T08:25:53.756 INFO:teuthology.orchestra.run.smithi031.stdout:4102 punch from 0x3c163b8 to 0x3c21155, (0xad9d bytes) 2024-05-08T08:25:53.756 INFO:teuthology.orchestra.run.smithi031.stdout:4103 read 0xd3503ce thru 0xd35d37b (0xcfae bytes) 2024-05-08T08:25:53.756 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.756 INFO:teuthology.orchestra.run.smithi031.stdout:4105 writesame 0x11f0735 thru 0x11f5d94 data_size 0x10(0x5660 bytes) 2024-05-08T08:25:53.761 INFO:teuthology.orchestra.run.smithi031.stdout:4107 read 0xd1e82de thru 0xd1f1247 (0x8f6a bytes) 2024-05-08T08:25:53.761 INFO:teuthology.orchestra.run.smithi031.stdout:4110 writesame 0x31b1020 thru 0x31b8c60 data_size 0x17(0x7c41 bytes) 2024-05-08T08:25:53.765 INFO:teuthology.orchestra.run.smithi031.stdout:4111 write 0x86db1b2 thru 0x86e106f (0x5ebe bytes) 2024-05-08T08:25:53.769 INFO:teuthology.orchestra.run.smithi031.stdout:4112 writesame 0xd087c2a thru 0xd097057 data_size 0x7(0xf42e bytes) 2024-05-08T08:25:53.775 INFO:teuthology.orchestra.run.smithi031.stdout:4113 read 0x5c44c2f thru 0x5c49ec5 (0x5297 bytes) 2024-05-08T08:25:53.775 INFO:teuthology.orchestra.run.smithi031.stdout:4115 punch from 0x8ad1ed to 0x8b3cff, (0x6b12 bytes) 2024-05-08T08:25:53.775 INFO:teuthology.orchestra.run.smithi031.stdout:4116 write 0x46fa99d thru 0x46fdda0 (0x3404 bytes) 2024-05-08T08:25:53.778 INFO:teuthology.orchestra.run.smithi031.stdout:4117 write 0xbb56b6d thru 0xbb5f8f9 (0x8d8d bytes) 2024-05-08T08:25:53.782 INFO:teuthology.orchestra.run.smithi031.stdout:4118 punch from 0x39a1d35 to 0x39aa23b, (0x8506 bytes) 2024-05-08T08:25:53.782 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.782 INFO:teuthology.orchestra.run.smithi031.stdout:4120 punch from 0x384992e to 0x3858a75, (0xf147 bytes) 2024-05-08T08:25:53.782 INFO:teuthology.orchestra.run.smithi031.stdout:4122 writesame 0x77642ca thru 0x7768539 data_size 0x8(0x4270 bytes) 2024-05-08T08:25:53.785 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:53.793+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:53.788 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.794 INFO:teuthology.orchestra.run.smithi031.stdout:4124 write 0xe7115d5 thru 0xe7207ef (0xf21b bytes) 2024-05-08T08:25:53.799 INFO:teuthology.orchestra.run.smithi031.stdout:4126 write 0xbfc951e thru 0xbfd53fc (0xbedf bytes) 2024-05-08T08:25:53.807 INFO:teuthology.orchestra.run.smithi031.stdout:4127 trunc from 0xe7207f0 to 0xaeff13d 2024-05-08T08:25:53.809 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:53.821 INFO:teuthology.orchestra.run.smithi031.stdout:4129 read 0x8c8713a thru 0x8c94584 (0xd44b bytes) 2024-05-08T08:25:53.821 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.821 INFO:teuthology.orchestra.run.smithi031.stdout:4135 read 0x6bcee2e thru 0x6bd1522 (0x26f5 bytes) 2024-05-08T08:25:53.821 INFO:teuthology.orchestra.run.smithi031.stdout:4136 read 0x65730eb thru 0x657ead6 (0xb9ec bytes) 2024-05-08T08:25:53.821 INFO:teuthology.orchestra.run.smithi031.stdout:4137 read 0x1f6bb7d thru 0x1f703fd (0x4881 bytes) 2024-05-08T08:25:53.821 INFO:teuthology.orchestra.run.smithi031.stdout:4138 read 0x450684a thru 0x4510529 (0x9ce0 bytes) 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:4139 punch from 0x34b0a4b to 0x34b5cec, (0x52a1 bytes) 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:4142 read 0xdea4ad thru 0xdefaa6 (0x55fa bytes) 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:4145 read 0x9f283e thru 0x9fb51c (0x8cdf bytes) 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:4146 punch from 0x3ae853a to 0x3af4692, (0xc158 bytes) 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:4147 read 0xa76e0fa thru 0xa770fca (0x2ed1 bytes) 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.822 INFO:teuthology.orchestra.run.smithi031.stdout:4149 writesame 0x1b9a819 thru 0x1ba1890 data_size 0x8(0x7078 bytes) 2024-05-08T08:25:53.825 INFO:teuthology.orchestra.run.smithi031.stdout:4150 read 0x8271757 thru 0x8273827 (0x20d1 bytes) 2024-05-08T08:25:53.825 INFO:teuthology.orchestra.run.smithi031.stdout:4151 writesame 0x4c9a893 thru 0x4ca6585 data_size 0x1(0xbcf3 bytes) 2024-05-08T08:25:53.840 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.841 INFO:teuthology.orchestra.run.smithi031.stdout:4153 write 0x2a193d5 thru 0x2a2645a (0xd086 bytes) 2024-05-08T08:25:53.846 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:53.846 INFO:teuthology.orchestra.run.smithi031.stdout:4155 write 0x22404ae thru 0x2240fbf (0xb12 bytes) 2024-05-08T08:25:53.852 INFO:teuthology.orchestra.run.smithi031.stdout:4158 punch from 0x471ca5 to 0x47d9bb, (0xbd16 bytes) 2024-05-08T08:25:53.852 INFO:teuthology.orchestra.run.smithi031.stdout:4159 read 0x3924df2 thru 0x3930e9b (0xc0aa bytes) 2024-05-08T08:25:53.852 INFO:teuthology.orchestra.run.smithi031.stdout:4162 writesame 0x8436e75 thru 0x8437425 data_size 0x1(0x5b1 bytes) 2024-05-08T08:25:53.859 INFO:teuthology.orchestra.run.smithi031.stdout:4163 trunc from 0xaeff13d to 0x984450c 2024-05-08T08:25:53.864 INFO:teuthology.orchestra.run.smithi031.stdout:4164 write 0x3133783 thru 0x313ee0c (0xb68a bytes) 2024-05-08T08:25:53.866 INFO:teuthology.orchestra.run.smithi031.stdout:4165 punch from 0x2f262c8 to 0x2f35661, (0xf399 bytes) 2024-05-08T08:25:53.866 INFO:teuthology.orchestra.run.smithi031.stdout:4166 read 0x65ab37e thru 0x65ac7bf (0x1442 bytes) 2024-05-08T08:25:53.868 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:53.868 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:53.868 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:53.868 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:53.869 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:53.873 INFO:teuthology.orchestra.run.smithi031.stdout:4167 write 0xa68edcf thru 0xa69acf3 (0xbf25 bytes) 2024-05-08T08:25:53.878 INFO:teuthology.orchestra.run.smithi031.stdout:4168 write 0x882494e thru 0x882503e (0x6f1 bytes) 2024-05-08T08:25:53.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:53.881 INFO:teuthology.orchestra.run.smithi031.stdout:4169 read 0x21d57c0 thru 0x21d6c12 (0x1453 bytes) 2024-05-08T08:25:53.881 INFO:teuthology.orchestra.run.smithi031.stdout:4170 read 0xe4fca3 thru 0xe5b2dd (0xb63b bytes) 2024-05-08T08:25:53.881 INFO:teuthology.orchestra.run.smithi031.stdout:4171 read 0x2a95bad thru 0x2a9daa0 (0x7ef4 bytes) 2024-05-08T08:25:53.884 INFO:teuthology.orchestra.run.smithi031.stdout:4172 trunc from 0xa69acf4 to 0x8027ff0 2024-05-08T08:25:53.886 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:53.893+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:53.896 INFO:teuthology.orchestra.run.smithi031.stdout:4173 read 0x361a276 thru 0x3628bf0 (0xe97b bytes) 2024-05-08T08:25:53.903 INFO:teuthology.orchestra.run.smithi031.stdout:4174 trunc from 0x8027ff0 to 0x2356c8f 2024-05-08T08:25:53.921 INFO:teuthology.orchestra.run.smithi031.stdout:4177 read 0x1e71c7 thru 0x1e9dbb (0x2bf5 bytes) 2024-05-08T08:25:53.940 INFO:teuthology.orchestra.run.smithi031.stdout:4178 write 0xd71108f thru 0xd71b932 (0xa8a4 bytes) 2024-05-08T08:25:53.946 INFO:teuthology.orchestra.run.smithi031.stdout:4179 write 0x504e633 thru 0x5054698 (0x6066 bytes) 2024-05-08T08:25:53.962 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:53.962 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:53.965 INFO:teuthology.orchestra.run.smithi031.stdout:4182 trunc from 0xd71b933 to 0x522d957 2024-05-08T08:25:53.982 INFO:teuthology.orchestra.run.smithi031.stdout:4183 writesame 0x532f9cd thru 0x533efb5 data_size 0x3b(0xf5e9 bytes) 2024-05-08T08:25:53.986 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:53.986 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:53.986 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:53.994+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:53.993 INFO:teuthology.orchestra.run.smithi031.stdout:4184 write 0x846b1bb thru 0x8475f14 (0xad5a bytes) 2024-05-08T08:25:53.995 INFO:teuthology.orchestra.run.smithi031.stdout:4190 read 0x655d74a thru 0x6567e54 (0xa70b bytes) 2024-05-08T08:25:53.995 INFO:teuthology.orchestra.run.smithi031.stdout:4192 punch from 0x13b137b to 0x13b5d8a, (0x4a0f bytes) 2024-05-08T08:25:53.995 INFO:teuthology.orchestra.run.smithi031.stdout:4193 read 0x6f261a3 thru 0x6f306f7 (0xa555 bytes) 2024-05-08T08:25:53.995 INFO:teuthology.orchestra.run.smithi031.stdout:4194 writesame 0x3e27428 thru 0x3e29cc3 data_size 0x17(0x289c bytes) 2024-05-08T08:25:53.998 INFO:teuthology.orchestra.run.smithi031.stdout:4195 read 0x53bf5b7 thru 0x53c468b (0x50d5 bytes) 2024-05-08T08:25:53.999 INFO:teuthology.orchestra.run.smithi031.stdout:4196 read 0x32e8e44 thru 0x32ef6b1 (0x686e bytes) 2024-05-08T08:25:54.005 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:54.005 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:54.008 INFO:teuthology.orchestra.run.smithi031.stdout:4197 write 0xb541433 thru 0xb547e51 (0x6a1f bytes) 2024-05-08T08:25:54.011 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.011 INFO:teuthology.orchestra.run.smithi031.stdout:4200 punch from 0x4f8c28e to 0x4f98318, (0xc08a bytes) 2024-05-08T08:25:54.013 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:54.013 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:54.018 INFO:teuthology.orchestra.run.smithi031.stdout:4201 writesame 0xc68bc29 thru 0xc696c43 data_size 0x1(0xb01b bytes) 2024-05-08T08:25:54.018 INFO:teuthology.orchestra.run.smithi031.stdout:4202 write 0x6bae744 thru 0x6bb0f5b (0x2818 bytes) 2024-05-08T08:25:54.024 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:54.025 INFO:teuthology.orchestra.run.smithi031.stdout:4203 write 0x5589be6 thru 0x558f956 (0x5d71 bytes) 2024-05-08T08:25:54.027 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:54.027 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:54.037 INFO:teuthology.orchestra.run.smithi031.stdout:4204 write 0xd75514a thru 0xd75e33b (0x91f2 bytes) 2024-05-08T08:25:54.043 INFO:teuthology.orchestra.run.smithi031.stdout:4207 write 0x67328ad thru 0x67419af (0xf103 bytes) 2024-05-08T08:25:54.048 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:54.048 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:54.048 INFO:teuthology.orchestra.run.smithi031.stdout:4208 write 0x72a86fc thru 0x72aeaa9 (0x63ae bytes) 2024-05-08T08:25:54.051 INFO:teuthology.orchestra.run.smithi031.stdout:4209 writesame 0xb5571e8 thru 0xb560950 data_size 0x1(0x9769 bytes) 2024-05-08T08:25:54.069 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:54.069 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:54.072 INFO:teuthology.orchestra.run.smithi031.stdout:4210 read 0x10a77e5 thru 0x10b529a (0xdab6 bytes) 2024-05-08T08:25:54.079 INFO:teuthology.orchestra.run.smithi031.stdout:4211 write 0xe997934 thru 0xe9a4c9a (0xd367 bytes) 2024-05-08T08:25:54.087 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:54.094+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:54.090 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:54.090 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:54.096 INFO:teuthology.orchestra.run.smithi031.stdout:4212 trunc from 0xe9a4c9b to 0x537950c 2024-05-08T08:25:54.113 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:54.113 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:54.116 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.121 INFO:teuthology.orchestra.run.smithi031.stdout:4214 trunc from 0x537950c to 0xb0610c 2024-05-08T08:25:54.147 INFO:teuthology.orchestra.run.smithi031.stdout:4215 trunc from 0xb0610c to 0xa856dbc 2024-05-08T08:25:54.153 INFO:teuthology.orchestra.run.smithi031.stdout:4216 writesame 0x2ec3892 thru 0x2ece404 data_size 0x1(0xab73 bytes) 2024-05-08T08:25:54.156 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:54.156 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:54.165 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:54.166 INFO:teuthology.orchestra.run.smithi031.stdout:4218 read 0x3d4551f thru 0x3d50306 (0xade8 bytes) 2024-05-08T08:25:54.166 INFO:teuthology.orchestra.run.smithi031.stdout:4219 read 0x1183376 thru 0x118b3b5 (0x8040 bytes) 2024-05-08T08:25:54.166 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.172 INFO:teuthology.orchestra.run.smithi031.stdout:4221 write 0xb37f9c5 thru 0xb38ab5a (0xb196 bytes) 2024-05-08T08:25:54.177 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.177 INFO:teuthology.orchestra.run.smithi031.stdout:4225 write 0x87ef1b8 thru 0x87f80aa (0x8ef3 bytes) 2024-05-08T08:25:54.181 INFO:teuthology.orchestra.run.smithi031.stdout:4226 write 0x9bc936f thru 0x9bd629e (0xcf30 bytes) 2024-05-08T08:25:54.185 INFO:teuthology.orchestra.run.smithi031.stdout:4227 read 0x7335d40 thru 0x7336f2b (0x11ec bytes) 2024-05-08T08:25:54.185 INFO:teuthology.orchestra.run.smithi031.stdout:4228 write 0x494befb thru 0x494e67e (0x2784 bytes) 2024-05-08T08:25:54.187 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:54.195+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:54.190 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:25:54.196 INFO:teuthology.orchestra.run.smithi031.stdout:4230 write 0xcc71f7d thru 0xcc797da (0x785e bytes) 2024-05-08T08:25:54.200 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.206 INFO:teuthology.orchestra.run.smithi031.stdout:4234 trunc from 0xcc797db to 0x78f9e8a 2024-05-08T08:25:54.218 INFO:teuthology.orchestra.run.smithi031.stdout:4235 trunc from 0x78f9e8a to 0x7259774 2024-05-08T08:25:54.224 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.224 INFO:teuthology.orchestra.run.smithi031.stdout:4238 read 0x41c0b9a thru 0x41d08cf (0xfd36 bytes) 2024-05-08T08:25:54.224 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.224 INFO:teuthology.orchestra.run.smithi031.stdout:4242 punch from 0x434c7e to 0x44428c, (0xf60e bytes) 2024-05-08T08:25:54.224 INFO:teuthology.orchestra.run.smithi031.stdout:4243 punch from 0x22d92d8 to 0x22e21af, (0x8ed7 bytes) 2024-05-08T08:25:54.231 INFO:teuthology.orchestra.run.smithi031.stdout:4244 write 0x76ea0c6 thru 0x76f86e8 (0xe623 bytes) 2024-05-08T08:25:54.235 INFO:teuthology.orchestra.run.smithi031.stdout:4245 write 0x3e49dc4 thru 0x3e56f04 (0xd141 bytes) 2024-05-08T08:25:54.239 INFO:teuthology.orchestra.run.smithi031.stdout:4246 punch from 0x432a8ff to 0x432e1d5, (0x38d6 bytes) 2024-05-08T08:25:54.239 INFO:teuthology.orchestra.run.smithi031.stdout:4247 read 0xf90b97 thru 0xf9254e (0x19b8 bytes) 2024-05-08T08:25:54.239 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.239 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.239 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.252 INFO:teuthology.orchestra.run.smithi031.stdout:4253 write 0xd5dc389 thru 0xd5e2dc9 (0x6a41 bytes) 2024-05-08T08:25:54.260 INFO:teuthology.orchestra.run.smithi031.stdout:4255 flatten 2024-05-08T08:25:54.289 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:54.296+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:54.300 INFO:teuthology.orchestra.run.smithi031.stdout:4256 read 0xb4abe3 thru 0xb4b030 (0x44e bytes) 2024-05-08T08:25:54.306 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:54.306 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:54.312 INFO:teuthology.orchestra.run.smithi031.stdout:4258 trunc from 0xd5e2dca to 0x4d0d753 2024-05-08T08:25:54.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:54.338 INFO:teuthology.orchestra.run.smithi031.stdout:4259 trunc from 0x4d0d753 to 0xe6e84f6 2024-05-08T08:25:54.343 INFO:teuthology.orchestra.run.smithi031.stdout:4260 writesame 0x638896b thru 0x6394ca0 data_size 0x3(0xc336 bytes) 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:4261 read 0x987e339 thru 0x988abac (0xc874 bytes) 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:4262 read 0xa28560a thru 0xa28cb6b (0x7562 bytes) 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:4263 read 0x935bb44 thru 0x9368d76 (0xd233 bytes) 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:4265 punch from 0x12aaa8e to 0x12ae9c1, (0x3f33 bytes) 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:4268 read 0x388970b thru 0x3897444 (0xdd3a bytes) 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.352 INFO:teuthology.orchestra.run.smithi031.stdout:4270 write 0x160e01e thru 0x161bcc7 (0xdcaa bytes) 2024-05-08T08:25:54.357 INFO:teuthology.orchestra.run.smithi031.stdout:4271 write 0x45618a3 thru 0x4569437 (0x7b95 bytes) 2024-05-08T08:25:54.365 INFO:teuthology.orchestra.run.smithi031.stdout:4272 writesame 0xcc1a33a thru 0xcc1ac09 data_size 0x2f(0x8d0 bytes) 2024-05-08T08:25:54.370 INFO:teuthology.orchestra.run.smithi031.stdout:4273 read 0x965e806 thru 0x9665afd (0x72f8 bytes) 2024-05-08T08:25:54.370 INFO:teuthology.orchestra.run.smithi031.stdout:4277 punch from 0xac77aed to 0xac7cedf, (0x53f2 bytes) 2024-05-08T08:25:54.370 INFO:teuthology.orchestra.run.smithi031.stdout:4279 write 0xd23ce55 thru 0xd24c6ce (0xf87a bytes) 2024-05-08T08:25:54.379 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.379 INFO:teuthology.orchestra.run.smithi031.stdout:4281 write 0x89132e2 thru 0x891c599 (0x92b8 bytes) 2024-05-08T08:25:54.387 INFO:teuthology.orchestra.run.smithi031.stdout:4282 read 0xbccf8f3 thru 0xbcd4ab0 (0x51be bytes) 2024-05-08T08:25:54.389 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:54.397+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:54.392 INFO:teuthology.orchestra.run.smithi031.stdout:4283 trunc from 0xe6e84f6 to 0xa1adbda 2024-05-08T08:25:54.407 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.411 INFO:teuthology.orchestra.run.smithi031.stdout:4287 trunc from 0xa1adbda to 0x66e1e65 2024-05-08T08:25:54.436 INFO:teuthology.orchestra.run.smithi031.stdout:4288 write 0xc8a20f8 thru 0xc8a49b6 (0x28bf bytes) 2024-05-08T08:25:54.439 INFO:teuthology.orchestra.run.smithi031.stdout:4290 writesame 0x611aa34 thru 0x611c57d data_size 0x7(0x1b4a bytes) 2024-05-08T08:25:54.443 INFO:teuthology.orchestra.run.smithi031.stdout:4291 write 0x53092f9 thru 0x5309a84 (0x78c bytes) 2024-05-08T08:25:54.455 INFO:teuthology.orchestra.run.smithi031.stdout:4293 write 0xda974eb thru 0xda9ca9f (0x55b5 bytes) 2024-05-08T08:25:54.458 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:54.458 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:54.460 INFO:teuthology.orchestra.run.smithi031.stdout:4294 write 0x2f1aee1 thru 0x2f1d68f (0x27af bytes) 2024-05-08T08:25:54.465 INFO:teuthology.orchestra.run.smithi031.stdout:4296 writesame 0xba0241e thru 0xba02c57 data_size 0x1b(0x83a bytes) 2024-05-08T08:25:54.469 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:54.470 INFO:teuthology.orchestra.run.smithi031.stdout:4298 writesame 0xb53d35e thru 0xb547cd5 data_size 0x58(0xa978 bytes) 2024-05-08T08:25:54.475 INFO:teuthology.orchestra.run.smithi031.stdout:4299 punch from 0x6d6163e to 0x6d70404, (0xedc6 bytes) 2024-05-08T08:25:54.475 INFO:teuthology.orchestra.run.smithi031.stdout:4300 write 0x8c573ef thru 0x8c5ecd3 (0x78e5 bytes) 2024-05-08T08:25:54.478 INFO:teuthology.orchestra.run.smithi031.stdout:4301 writesame 0xce22062 thru 0xce2ee10 data_size 0x1(0xcdaf bytes) 2024-05-08T08:25:54.494 INFO:teuthology.orchestra.run.smithi031.stdout:4303 trunc from 0xda9caa0 to 0xdbcf52d 2024-05-08T08:25:54.499 INFO:teuthology.orchestra.run.smithi031.stdout:4304 write 0xae28440 thru 0xae28dad (0x96e bytes) 2024-05-08T08:25:54.503 INFO:teuthology.orchestra.run.smithi031.stdout:4307 read 0x2f7996 thru 0x307123 (0xf78e bytes) 2024-05-08T08:25:54.503 INFO:teuthology.orchestra.run.smithi031.stdout:4309 write 0xd8695c3 thru 0xd86ed98 (0x57d6 bytes) 2024-05-08T08:25:54.506 INFO:teuthology.orchestra.run.smithi031.stdout:4310 write 0x6801d42 thru 0x6807873 (0x5b32 bytes) 2024-05-08T08:25:54.509 INFO:teuthology.orchestra.run.smithi031.stdout:4312 write 0x78aca3b thru 0x78b45c1 (0x7b87 bytes) 2024-05-08T08:25:54.513 INFO:teuthology.orchestra.run.smithi031.stdout:4313 write 0x839b2bd thru 0x83a2d96 (0x7ada bytes) 2024-05-08T08:25:54.513 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:54.521+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:54.516 INFO:teuthology.orchestra.run.smithi031.stdout:4316 punch from 0xaba89f0 to 0xabae3b1, (0x59c1 bytes) 2024-05-08T08:25:54.517 INFO:teuthology.orchestra.run.smithi031.stdout:4318 punch from 0xcd559a9 to 0xcd60e73, (0xb4ca bytes) 2024-05-08T08:25:54.517 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.517 INFO:teuthology.orchestra.run.smithi031.stdout:4323 write 0xc220a2e thru 0xc221b9e (0x1171 bytes) 2024-05-08T08:25:54.534 INFO:teuthology.orchestra.run.smithi031.stdout:4324 trunc from 0xdbcf52d to 0x4d384d 2024-05-08T08:25:54.561 INFO:teuthology.orchestra.run.smithi031.stdout:4325 writesame 0x438f00b thru 0x439b6fb data_size 0x1(0xc6f1 bytes) 2024-05-08T08:25:54.568 INFO:teuthology.orchestra.run.smithi031.stdout:4326 trunc from 0x439b6fc to 0xa9a756f 2024-05-08T08:25:54.577 INFO:teuthology.orchestra.run.smithi031.stdout:4328 trunc from 0xa9a756f to 0xd73d400 2024-05-08T08:25:54.582 INFO:teuthology.orchestra.run.smithi031.stdout:4330 read 0x7bca824 thru 0x7bd4451 (0x9c2e bytes) 2024-05-08T08:25:54.582 INFO:teuthology.orchestra.run.smithi031.stdout:4331 write 0x7ec99ad thru 0x7ed4520 (0xab74 bytes) 2024-05-08T08:25:54.582 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:54.583 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:54.586 INFO:teuthology.orchestra.run.smithi031.stdout:4334 read 0x75ae266 thru 0x75b48b6 (0x6651 bytes) 2024-05-08T08:25:54.586 INFO:teuthology.orchestra.run.smithi031.stdout:4335 read 0x4c276bc thru 0x4c2b56a (0x3eaf bytes) 2024-05-08T08:25:54.587 INFO:teuthology.orchestra.run.smithi031.stdout:4336 punch from 0x776a004 to 0x7778ac0, (0xeabc bytes) 2024-05-08T08:25:54.587 INFO:teuthology.orchestra.run.smithi031.stdout:4337 read 0x2cb0774 thru 0x2cb62f5 (0x5b82 bytes) 2024-05-08T08:25:54.587 INFO:teuthology.orchestra.run.smithi031.stdout:4338 punch from 0x34e4b36 to 0x34eeac4, (0x9f8e bytes) 2024-05-08T08:25:54.587 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.588 INFO:teuthology.orchestra.run.smithi031.stdout:4343 trunc from 0xd73d400 to 0xb472906 2024-05-08T08:25:54.592 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:54.592 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:54.599+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:54.594 INFO:teuthology.orchestra.run.smithi031.stdout:4345 write 0xb1382cc thru 0xb138f0e (0xc43 bytes) 2024-05-08T08:25:54.597 INFO:teuthology.orchestra.run.smithi031.stdout:4346 writesame 0x341f753 thru 0x3428608 data_size 0x3(0x8eb6 bytes) 2024-05-08T08:25:54.604 INFO:teuthology.orchestra.run.smithi031.stdout:4347 writesame 0x4d58e07 thru 0x4d639e7 data_size 0x3(0xabe1 bytes) 2024-05-08T08:25:54.613 INFO:teuthology.orchestra.run.smithi031.stdout:4348 read 0x70f509e thru 0x70f87b7 (0x371a bytes) 2024-05-08T08:25:54.613 INFO:teuthology.orchestra.run.smithi031.stdout:4350 write 0x87f7762 thru 0x8806bdf (0xf47e bytes) 2024-05-08T08:25:54.621 INFO:teuthology.orchestra.run.smithi031.stdout:4351 write 0x7861ef thru 0x78edf9 (0x8c0b bytes) 2024-05-08T08:25:54.635 INFO:teuthology.orchestra.run.smithi031.stdout:4353 write 0xdd81046 thru 0xdd89b77 (0x8b32 bytes) 2024-05-08T08:25:54.638 INFO:teuthology.orchestra.run.smithi031.stdout:4354 read 0xc95308f thru 0xc9593e1 (0x6353 bytes) 2024-05-08T08:25:54.639 INFO:teuthology.orchestra.run.smithi031.stdout:4355 trunc from 0xdd89b78 to 0xe59389b 2024-05-08T08:25:54.644 INFO:teuthology.orchestra.run.smithi031.stdout:4357 write 0xa545b5e thru 0xa546a3b (0xede bytes) 2024-05-08T08:25:54.646 INFO:teuthology.orchestra.run.smithi031.stdout:4358 read 0x318964 thru 0x324963 (0xc000 bytes) 2024-05-08T08:25:54.647 INFO:teuthology.orchestra.run.smithi031.stdout:4360 writesame 0xc7dc2db thru 0xc7dc9f2 data_size 0x4(0x718 bytes) 2024-05-08T08:25:54.649 INFO:teuthology.orchestra.run.smithi031.stdout:4361 read 0x7fec793 thru 0x7fed115 (0x983 bytes) 2024-05-08T08:25:54.649 INFO:teuthology.orchestra.run.smithi031.stdout:4362 read 0xcf68b19 thru 0xcf70d57 (0x823f bytes) 2024-05-08T08:25:54.649 INFO:teuthology.orchestra.run.smithi031.stdout:4363 write 0xc549a6e thru 0xc55552f (0xbac2 bytes) 2024-05-08T08:25:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:4365 punch from 0x935afd9 to 0x935f54f, (0x4576 bytes) 2024-05-08T08:25:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:4370 read 0xa135a11 thru 0xa140e0a (0xb3fa bytes) 2024-05-08T08:25:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:4372 punch from 0x28f54fc to 0x28f908b, (0x3b8f bytes) 2024-05-08T08:25:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:4373 punch from 0x196b196 to 0x1974fe2, (0x9e4c bytes) 2024-05-08T08:25:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:4374 punch from 0xe40c3b1 to 0xe40c810, (0x45f bytes) 2024-05-08T08:25:54.656 INFO:teuthology.orchestra.run.smithi031.stdout:4376 writesame 0xe4ce9c8 thru 0xe4ddf5e data_size 0x13(0xf597 bytes) 2024-05-08T08:25:54.664 INFO:teuthology.orchestra.run.smithi031.stdout:4378 write 0x8a6689 thru 0x8acdd9 (0x6751 bytes) 2024-05-08T08:25:54.673 INFO:teuthology.orchestra.run.smithi031.stdout:4379 trunc from 0xe59389b to 0xe7c6ac3 2024-05-08T08:25:54.679 INFO:teuthology.orchestra.run.smithi031.stdout:4380 write 0x91332a4 thru 0x91429a8 (0xf705 bytes) 2024-05-08T08:25:54.687 INFO:teuthology.orchestra.run.smithi031.stdout:4381 write 0x8a96701 thru 0x8a9811d (0x1a1d bytes) 2024-05-08T08:25:54.692 INFO:teuthology.orchestra.run.smithi031.stdout:4382 punch from 0xe28c00a to 0xe28f819, (0x380f bytes) 2024-05-08T08:25:54.692 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.692 INFO:teuthology.orchestra.run.smithi031.stdout:4386 punch from 0xc780983 to 0xc79067e, (0xfcfb bytes) 2024-05-08T08:25:54.693 INFO:teuthology.orchestra.run.smithi031.stdout:4387 write 0x6697a07 thru 0x669a4d1 (0x2acb bytes) 2024-05-08T08:25:54.693 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:54.700+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:54.696 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.696 INFO:teuthology.orchestra.run.smithi031.stdout:4389 writesame 0x2745580 thru 0x2747466 data_size 0x9(0x1ee7 bytes) 2024-05-08T08:25:54.711 INFO:teuthology.orchestra.run.smithi031.stdout:4390 trunc from 0xe7c6ac3 to 0x469e1b5 2024-05-08T08:25:54.745 INFO:teuthology.orchestra.run.smithi031.stdout:4392 write 0xafbc812 thru 0xafc65f4 (0x9de3 bytes) 2024-05-08T08:25:54.748 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:54.748 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:54.749 INFO:teuthology.orchestra.run.smithi031.stdout:4393 read 0x844647 thru 0x852109 (0xdac3 bytes) 2024-05-08T08:25:54.749 INFO:teuthology.orchestra.run.smithi031.stdout:4394 writesame 0x9b815ad thru 0x9b8e314 data_size 0x54(0xcd68 bytes) 2024-05-08T08:25:54.754 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.758 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:54.760 INFO:teuthology.orchestra.run.smithi031.stdout:4396 write 0xb64c9af thru 0xb65a04c (0xd69e bytes) 2024-05-08T08:25:54.771 INFO:teuthology.orchestra.run.smithi031.stdout:4397 write 0x1be3a7c thru 0x1beec52 (0xb1d7 bytes) 2024-05-08T08:25:54.777 INFO:teuthology.orchestra.run.smithi031.stdout:4398 read 0x958e591 thru 0x95929c5 (0x4435 bytes) 2024-05-08T08:25:54.778 INFO:teuthology.orchestra.run.smithi031.stdout:4399 trunc from 0xb65a04d to 0xaba48cc 2024-05-08T08:25:54.788 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.788 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.788 INFO:teuthology.orchestra.run.smithi031.stdout:4402 write 0x4c96a4f thru 0x4ca4fd0 (0xe582 bytes) 2024-05-08T08:25:54.793 INFO:teuthology.orchestra.run.smithi031.stdout:4404 write 0x426169a thru 0x4261f9d (0x904 bytes) 2024-05-08T08:25:54.796 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.796 INFO:teuthology.orchestra.run.smithi031.stdout:4409 read 0x3850963 thru 0x385ad13 (0xa3b1 bytes) 2024-05-08T08:25:54.796 INFO:teuthology.orchestra.run.smithi031.stdout:4410 punch from 0x16fdb10 to 0x16fe027, (0x517 bytes) 2024-05-08T08:25:54.799 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:54.807+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:54.802 INFO:teuthology.orchestra.run.smithi031.stdout:4412 write 0xbe022c7 thru 0xbe0c38f (0xa0c9 bytes) 2024-05-08T08:25:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:4413 read 0x3a7fedb thru 0x3a8928d (0x93b3 bytes) 2024-05-08T08:25:54.806 INFO:teuthology.orchestra.run.smithi031.stdout:4414 punch from 0x47bc093 to 0x47c5ed8, (0x9e45 bytes) 2024-05-08T08:25:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:4416 read 0x1498262 thru 0x14a5a59 (0xd7f8 bytes) 2024-05-08T08:25:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:4418 read 0x9883327 thru 0x988f22b (0xbf05 bytes) 2024-05-08T08:25:54.807 INFO:teuthology.orchestra.run.smithi031.stdout:4419 write 0x5d9642e thru 0x5d9b29b (0x4e6e bytes) 2024-05-08T08:25:54.810 INFO:teuthology.orchestra.run.smithi031.stdout:4420 writesame 0x7d4496c thru 0x7d4e45a data_size 0x71(0x9aef bytes) 2024-05-08T08:25:54.814 INFO:teuthology.orchestra.run.smithi031.stdout:4421 read 0x4383025 thru 0x438ae0a (0x7de6 bytes) 2024-05-08T08:25:54.814 INFO:teuthology.orchestra.run.smithi031.stdout:4422 punch from 0x42c2a28 to 0x42cf123, (0xc6fb bytes) 2024-05-08T08:25:54.814 INFO:teuthology.orchestra.run.smithi031.stdout:4423 writesame 0x50787b7 thru 0x507d5d4 data_size 0x65(0x4e1e bytes) 2024-05-08T08:25:54.817 INFO:teuthology.orchestra.run.smithi031.stdout:4424 write 0x2678b7 thru 0x26fb7c (0x82c6 bytes) 2024-05-08T08:25:54.821 INFO:teuthology.orchestra.run.smithi031.stdout:4425 writesame 0xcc1a4e thru 0xccd565 data_size 0x4(0xbb18 bytes) 2024-05-08T08:25:54.828 INFO:teuthology.orchestra.run.smithi031.stdout:4426 read 0x26a3d4b thru 0x26a8304 (0x45ba bytes) 2024-05-08T08:25:54.829 INFO:teuthology.orchestra.run.smithi031.stdout:4427 read 0x1646e0d thru 0x1647c27 (0xe1b bytes) 2024-05-08T08:25:54.829 INFO:teuthology.orchestra.run.smithi031.stdout:4428 writesame 0xb1c476d thru 0xb1cc34b data_size 0x1(0x7bdf bytes) 2024-05-08T08:25:54.840 INFO:teuthology.orchestra.run.smithi031.stdout:4430 read 0x44de64 thru 0x45612a (0x82c7 bytes) 2024-05-08T08:25:54.850 INFO:teuthology.orchestra.run.smithi031.stdout:4431 trunc from 0xbe0c390 to 0x36f10bc 2024-05-08T08:25:54.862 INFO:teuthology.orchestra.run.smithi031.stdout:4434 punch from 0x13fbd81 to 0x1403008, (0x7287 bytes) 2024-05-08T08:25:54.871 INFO:teuthology.orchestra.run.smithi031.stdout:4435 write 0x6d44f59 thru 0x6d463a2 (0x144a bytes) 2024-05-08T08:25:54.875 INFO:teuthology.orchestra.run.smithi031.stdout:4437 punch from 0x1db2cb6 to 0x1db303e, (0x388 bytes) 2024-05-08T08:25:54.875 INFO:teuthology.orchestra.run.smithi031.stdout:4438 punch from 0x46f5acb to 0x46fc0b7, (0x65ec bytes) 2024-05-08T08:25:54.875 INFO:teuthology.orchestra.run.smithi031.stdout:4439 read 0x2ff8a94 thru 0x2ff9894 (0xe01 bytes) 2024-05-08T08:25:54.882 INFO:teuthology.orchestra.run.smithi031.stdout:4444 write 0x8a46867 thru 0x8a4e01f (0x77b9 bytes) 2024-05-08T08:25:54.888 INFO:teuthology.orchestra.run.smithi031.stdout:4446 read 0x502821f thru 0x50306ac (0x848e bytes) 2024-05-08T08:25:54.894 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:54.901+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:54.895 INFO:teuthology.orchestra.run.smithi031.stdout:4447 write 0xa300746 thru 0xa30d668 (0xcf23 bytes) 2024-05-08T08:25:54.902 INFO:teuthology.orchestra.run.smithi031.stdout:4448 write 0x6592c6 thru 0x66770f (0xe44a bytes) 2024-05-08T08:25:54.912 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:54.912 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:54.915 INFO:teuthology.orchestra.run.smithi031.stdout:4449 write 0xa329186 thru 0xa32e186 (0x5001 bytes) 2024-05-08T08:25:54.918 INFO:teuthology.orchestra.run.smithi031.stdout:4450 read 0x99faec0 thru 0x9a06cab (0xbdec bytes) 2024-05-08T08:25:54.918 INFO:teuthology.orchestra.run.smithi031.stdout:4451 punch from 0x257719f to 0x257fb95, (0x89f6 bytes) 2024-05-08T08:25:54.918 INFO:teuthology.orchestra.run.smithi031.stdout:4453 writesame 0x19370aa thru 0x1946f36 data_size 0x1(0xfe8d bytes) 2024-05-08T08:25:54.922 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:54.935 INFO:teuthology.orchestra.run.smithi031.stdout:4454 read 0x301744e thru 0x302329b (0xbe4e bytes) 2024-05-08T08:25:54.936 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.938 INFO:teuthology.orchestra.run.smithi031.stdout:4456 trunc from 0xa32e187 to 0xca25195 2024-05-08T08:25:54.944 INFO:teuthology.orchestra.run.smithi031.stdout:4457 writesame 0x2b92f35 thru 0x2b9e178 data_size 0x6(0xb244 bytes) 2024-05-08T08:25:54.950 INFO:teuthology.orchestra.run.smithi031.stdout:4459 read 0x8aa6ad6 thru 0x8aa7712 (0xc3d bytes) 2024-05-08T08:25:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:4461 read 0x1c4c850 thru 0x1c55e7c (0x962d bytes) 2024-05-08T08:25:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:4463 read 0xc14864e thru 0xc14ad2d (0x26e0 bytes) 2024-05-08T08:25:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:4464 punch from 0x8f18ce6 to 0x8f273cc, (0xe6e6 bytes) 2024-05-08T08:25:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:4465 read 0x9ca1011 thru 0x9ca26ff (0x16ef bytes) 2024-05-08T08:25:54.951 INFO:teuthology.orchestra.run.smithi031.stdout:4467 write 0x467e892 thru 0x467ed75 (0x4e4 bytes) 2024-05-08T08:25:54.953 INFO:teuthology.orchestra.run.smithi031.stdout:4468 read 0xafdf546 thru 0xafe1ac8 (0x2583 bytes) 2024-05-08T08:25:54.953 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.953 INFO:teuthology.orchestra.run.smithi031.stdout:4470 read 0x13045b2 thru 0x13144e2 (0xff31 bytes) 2024-05-08T08:25:54.964 INFO:teuthology.orchestra.run.smithi031.stdout:4471 trunc from 0xca25195 to 0x30c9f54 2024-05-08T08:25:54.976 INFO:teuthology.orchestra.run.smithi031.stdout:4473 read 0x1d12c00 thru 0x1d13643 (0xa44 bytes) 2024-05-08T08:25:54.976 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.976 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:54.987 INFO:teuthology.orchestra.run.smithi031.stdout:4478 writesame 0x8358652 thru 0x8360598 data_size 0x1(0x7f47 bytes) 2024-05-08T08:25:54.987 INFO:teuthology.orchestra.run.smithi031.stdout:4479 read 0x2e452a9 thru 0x2e492ee (0x4046 bytes) 2024-05-08T08:25:54.987 INFO:teuthology.orchestra.run.smithi031.stdout:4480 writesame 0x22b3caa thru 0x22b80e5 data_size 0x16(0x443c bytes) 2024-05-08T08:25:54.995 INFO:teuthology.orchestra.run.smithi031.stdout:4481 trunc from 0x8360599 to 0xc302536 2024-05-08T08:25:54.995 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:55.003+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:55.001 INFO:teuthology.orchestra.run.smithi031.stdout:4483 read 0x3b3876 thru 0x3c0dc1 (0xd54c bytes) 2024-05-08T08:25:55.006 INFO:teuthology.orchestra.run.smithi031.stdout:4484 trunc from 0xc302536 to 0x79a215f 2024-05-08T08:25:55.012 INFO:teuthology.orchestra.run.smithi031.stdout:4486 trunc from 0x79a215f to 0x73f618f 2024-05-08T08:25:55.017 INFO:teuthology.orchestra.run.smithi031.stdout:4487 read 0x11a9af5 thru 0x11b3c46 (0xa152 bytes) 2024-05-08T08:25:55.017 INFO:teuthology.orchestra.run.smithi031.stdout:4488 trunc from 0x73f618f to 0x7c67091 2024-05-08T08:25:55.033 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.033 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.033 INFO:teuthology.orchestra.run.smithi031.stdout:4489 write 0xd8501d4 thru 0xd85ab85 (0xa9b2 bytes) 2024-05-08T08:25:55.037 INFO:teuthology.orchestra.run.smithi031.stdout:4490 write 0x3b3bcda thru 0x3b4b9cc (0xfcf3 bytes) 2024-05-08T08:25:55.042 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:55.042 INFO:teuthology.orchestra.run.smithi031.stdout:4496 write 0x23b3f27 thru 0x23c1fa8 (0xe082 bytes) 2024-05-08T08:25:55.044 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:55.045 INFO:teuthology.orchestra.run.smithi031.stdout:4497 writesame 0xcb3b7b6 thru 0xcb3f7d9 data_size 0xa(0x4024 bytes) 2024-05-08T08:25:55.049 INFO:teuthology.orchestra.run.smithi031.stdout:4501 read 0x396cfa7 thru 0x396eed3 (0x1f2d bytes) 2024-05-08T08:25:55.049 INFO:teuthology.orchestra.run.smithi031.stdout:4502 write 0x1bb15b4 thru 0x1bb4da3 (0x37f0 bytes) 2024-05-08T08:25:55.053 INFO:teuthology.orchestra.run.smithi031.stdout:4503 write 0xaec00d3 thru 0xaece76b (0xe699 bytes) 2024-05-08T08:25:55.060 INFO:teuthology.orchestra.run.smithi031.stdout:4504 punch from 0x1522cb6 to 0x152bdde, (0x9128 bytes) 2024-05-08T08:25:55.060 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:55.060 INFO:teuthology.orchestra.run.smithi031.stdout:4506 write 0x6b6463 thru 0x6bf76a (0x9308 bytes) 2024-05-08T08:25:55.067 INFO:teuthology.orchestra.run.smithi031.stdout:4507 punch from 0x702417a to 0x7027787, (0x360d bytes) 2024-05-08T08:25:55.069 INFO:teuthology.orchestra.run.smithi031.stdout:4508 trunc from 0xd85ab86 to 0xbc161be 2024-05-08T08:25:55.084 INFO:teuthology.orchestra.run.smithi031.stdout:4510 write 0x90bc8f thru 0x917559 (0xb8cb bytes) 2024-05-08T08:25:55.089 INFO:teuthology.orchestra.run.smithi031.stdout:4514 read 0x98bb6d9 thru 0x98bc066 (0x98e bytes) 2024-05-08T08:25:55.089 INFO:teuthology.orchestra.run.smithi031.stdout:4515 write 0x25a382b thru 0x25afd5c (0xc532 bytes) 2024-05-08T08:25:55.092 INFO:teuthology.orchestra.run.smithi031.stdout:4517 read 0x8b30117 thru 0x8b317d6 (0x16c0 bytes) 2024-05-08T08:25:55.093 INFO:teuthology.orchestra.run.smithi031.stdout:4518 write 0x4bf95cc thru 0x4c03b3e (0xa573 bytes) 2024-05-08T08:25:55.095 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:55.103+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:55.108 INFO:teuthology.orchestra.run.smithi031.stdout:4519 trunc from 0xbc161be to 0xfa30eb 2024-05-08T08:25:55.139 INFO:teuthology.orchestra.run.smithi031.stdout:4520 write 0x7074e82 thru 0x7080de6 (0xbf65 bytes) 2024-05-08T08:25:55.154 INFO:teuthology.orchestra.run.smithi031.stdout:4521 write 0xa791693 thru 0xa797faa (0x6918 bytes) 2024-05-08T08:25:55.157 INFO:teuthology.orchestra.run.smithi031.stdout:4522 write 0x2e092b5 thru 0x2e1383b (0xa587 bytes) 2024-05-08T08:25:55.160 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.160 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.163 INFO:teuthology.orchestra.run.smithi031.stdout:4524 read 0x210cc6b thru 0x210ecd6 (0x206c bytes) 2024-05-08T08:25:55.163 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:55.170 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:55.173 INFO:teuthology.orchestra.run.smithi031.stdout:4526 trunc from 0xa797fab to 0x175af05 2024-05-08T08:25:55.183 INFO:teuthology.orchestra.run.smithi031.stdout:4527 punch from 0xbc24ac to 0xbcc54f, (0xa0a3 bytes) 2024-05-08T08:25:55.183 INFO:teuthology.orchestra.run.smithi031.stdout:4530 read 0x8bae66 thru 0x8c7f98 (0xd133 bytes) 2024-05-08T08:25:55.183 INFO:teuthology.orchestra.run.smithi031.stdout:4531 write 0xde1614 thru 0xdf056b (0xef58 bytes) 2024-05-08T08:25:55.188 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:25:55.197 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:55.204+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:55.207 INFO:teuthology.orchestra.run.smithi031.stdout:4532 write 0xa8b084d thru 0xa8b3f10 (0x36c4 bytes) 2024-05-08T08:25:55.213 INFO:teuthology.orchestra.run.smithi031.stdout:4533 trunc from 0xa8b3f11 to 0x8bab4b0 2024-05-08T08:25:55.221 INFO:teuthology.orchestra.run.smithi031.stdout:4535 writesame 0x35fb962 thru 0x35fc572 data_size 0x1(0xc11 bytes) 2024-05-08T08:25:55.226 INFO:teuthology.orchestra.run.smithi031.stdout:4537 punch from 0x3b5db12 to 0x3b6d7d4, (0xfcc2 bytes) 2024-05-08T08:25:55.226 INFO:teuthology.orchestra.run.smithi031.stdout:4538 punch from 0x147dd42 to 0x1481a07, (0x3cc5 bytes) 2024-05-08T08:25:55.226 INFO:teuthology.orchestra.run.smithi031.stdout:4540 write 0x62e0c86 thru 0x62ee379 (0xd6f4 bytes) 2024-05-08T08:25:55.230 INFO:teuthology.orchestra.run.smithi031.stdout:4541 write 0x6420a87 thru 0x64270a2 (0x661c bytes) 2024-05-08T08:25:55.234 INFO:teuthology.orchestra.run.smithi031.stdout:4542 read 0x1339445 thru 0x133cd04 (0x38c0 bytes) 2024-05-08T08:25:55.234 INFO:teuthology.orchestra.run.smithi031.stdout:4543 write 0x612c456 thru 0x6136bd9 (0xa784 bytes) 2024-05-08T08:25:55.239 INFO:teuthology.orchestra.run.smithi031.stdout:4544 read 0x7d80730 thru 0x7d8772b (0x6ffc bytes) 2024-05-08T08:25:55.239 INFO:teuthology.orchestra.run.smithi031.stdout:4546 punch from 0x6385323 to 0x638c24f, (0x6f2c bytes) 2024-05-08T08:25:55.239 INFO:teuthology.orchestra.run.smithi031.stdout:4548 write 0x12f40e2 thru 0x12feccc (0xabeb bytes) 2024-05-08T08:25:55.256 INFO:teuthology.orchestra.run.smithi031.stdout:4549 write 0xdb561d6 thru 0xdb65d21 (0xfb4c bytes) 2024-05-08T08:25:55.260 INFO:teuthology.orchestra.run.smithi031.stdout:4551 read 0x795827e thru 0x795fecb (0x7c4e bytes) 2024-05-08T08:25:55.260 INFO:teuthology.orchestra.run.smithi031.stdout:4552 clone 26 order 19 su 32768 sc 6 2024-05-08T08:25:55.298 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:55.305+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:55.318 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.318 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.327 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:55.399 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:55.407+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:55.434 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.435 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.444 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:55.528 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:55.536+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:55.590 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.590 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.599 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:55.641 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:55.648+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:55.703 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:55.710+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:55.746 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.746 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.755 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:55.803 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:25:55.810+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:25:55.861 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.862 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.870 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:55.904 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:55.912+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:55.975 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:55.975 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:55.984 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:56.004 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:56.012+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:56.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:25:56.089 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:56.089 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:56.098 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:56.105 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:56.112+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:56.128 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:25:56.206 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:56.213+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:56.232 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:56.232 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:56.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:25:56.307 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:56.314+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:56.343 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:56.343 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:56.364 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:56.364 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:56.364 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:56.364 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:56.365 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:56.365 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:56.375 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:25:56.390 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:56.391 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:56.408 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:56.415+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:56.412 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:56.412 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:56.414 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-05-08T08:25:56.431 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:56.431 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:56.450 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:56.450 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:56.458 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:25:56.459 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:25:56.470 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:56.471 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:56.472 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-05-08T08:25:56.430899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":930204218,"num_objects":1291,"num_object_clones":265,"num_object_copies":2582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1291,"num_whiteouts":464,"num_read":90859,"num_read_kb":648643,"num_write":15178,"num_write_kb":374537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":14928057,"num_keys_recovered":23,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":11726,"ondisk_log_size":11726,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":363612,"kb_used_data":55596,"kb_used_omap":24,"kb_used_meta":307879,"kb_avail":754611108,"statfs":{"total":773094113280,"available":772721774592,"internally_reserved":0,"allocated":56930304,"data_stored":561575470,"data_compressed":18200461,"data_compressed_allocated":42983424,"data_compressed_original":554113177,"omap_allocated":24704,"internal_metadata":315268992},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,16],"upper_bound":8},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34745247,"num_objects":167,"num_object_clones":4,"num_object_copies":334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":167,"num_whiteouts":2,"num_read":30581,"num_read_kb":38436,"num_write":2947,"num_write_kb":21148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":1182817,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.252314"},"pg_stats":[{"pgid":"3.1f","version":"64'415","reported_seq":2396,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.269282+0000","last_change":"2024-05-08T08:25:06.628835+0000","last_active":"2024-05-08T08:25:55.269282+0000","last_peered":"2024-05-08T08:25:55.269282+0000","last_clean":"2024-05-08T08:25:55.269282+0000","last_became_active":"2024-05-08T08:25:04.610836+0000","last_became_peered":"2024-05-08T08:25:04.610836+0000","last_unstale":"2024-05-08T08:25:55.269282+0000","last_undegraded":"2024-05-08T08:25:55.269282+0000","last_fullsized":"2024-05-08T08:25:55.269282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:42:16.989054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011177400000000001,"stat_sum":{"num_bytes":21048515,"num_objects":36,"num_object_clones":6,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":4732,"num_read_kb":16856,"num_write":402,"num_write_kb":6748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"64'668","reported_seq":2573,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.198180+0000","last_change":"2024-05-08T08:25:06.611278+0000","last_active":"2024-05-08T08:25:55.198180+0000","last_peered":"2024-05-08T08:25:55.198180+0000","last_clean":"2024-05-08T08:25:55.198180+0000","last_became_active":"2024-05-08T08:25:04.611994+0000","last_became_peered":"2024-05-08T08:25:04.611994+0000","last_unstale":"2024-05-08T08:25:55.198180+0000","last_undegraded":"2024-05-08T08:25:55.198180+0000","last_fullsized":"2024-05-08T08:25:55.198180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:22:22.856794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010168530000000001,"stat_sum":{"num_bytes":21155525,"num_objects":55,"num_object_clones":13,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":22,"num_read":4501,"num_read_kb":24795,"num_write":1106,"num_write_kb":17289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"63'243","reported_seq":1158,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.197778+0000","last_change":"2024-05-08T08:25:06.625063+0000","last_active":"2024-05-08T08:25:55.197778+0000","last_peered":"2024-05-08T08:25:55.197778+0000","last_clean":"2024-05-08T08:25:55.197778+0000","last_became_active":"2024-05-08T08:25:04.610134+0000","last_became_peered":"2024-05-08T08:25:04.610134+0000","last_unstale":"2024-05-08T08:25:55.197778+0000","last_undegraded":"2024-05-08T08:25:55.197778+0000","last_fullsized":"2024-05-08T08:25:55.197778+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":243,"log_dups_size":0,"ondisk_log_size":243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:11:21.845648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005171,"stat_sum":{"num_bytes":14803060,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":1270,"num_read_kb":17841,"num_write":259,"num_write_kb":10038,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"63'248","reported_seq":1908,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.197700+0000","last_change":"2024-05-08T08:25:06.625044+0000","last_active":"2024-05-08T08:25:55.197700+0000","last_peered":"2024-05-08T08:25:55.197700+0000","last_clean":"2024-05-08T08:25:55.197700+0000","last_became_active":"2024-05-08T08:25:04.615120+0000","last_became_peered":"2024-05-08T08:25:04.615120+0000","last_unstale":"2024-05-08T08:25:55.197700+0000","last_undegraded":"2024-05-08T08:25:55.197700+0000","last_fullsized":"2024-05-08T08:25:55.197700+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:10.805571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010445210000000001,"stat_sum":{"num_bytes":95123675,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":1984,"num_read_kb":56806,"num_write":305,"num_write_kb":21914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"63'954","reported_seq":1507,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.209794+0000","last_change":"2024-05-08T08:25:06.628198+0000","last_active":"2024-05-08T08:25:55.209794+0000","last_peered":"2024-05-08T08:25:55.209794+0000","last_clean":"2024-05-08T08:25:55.209794+0000","last_became_active":"2024-05-08T08:25:04.612215+0000","last_became_peered":"2024-05-08T08:25:04.612215+0000","last_unstale":"2024-05-08T08:25:55.209794+0000","last_undegraded":"2024-05-08T08:25:55.209794+0000","last_fullsized":"2024-05-08T08:25:55.209794+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":954,"log_dups_size":0,"ondisk_log_size":954,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:10:41.801768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064530100000000003,"stat_sum":{"num_bytes":18497008,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":20,"num_read":5232,"num_read_kb":5326,"num_write":1564,"num_write_kb":2685,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"63'304","reported_seq":4893,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.229748+0000","last_change":"2024-05-08T08:25:48.158059+0000","last_active":"2024-05-08T08:25:55.229748+0000","last_peered":"2024-05-08T08:25:55.229748+0000","last_clean":"2024-05-08T08:25:55.229748+0000","last_became_active":"2024-05-08T08:25:48.157597+0000","last_became_peered":"2024-05-08T08:25:48.157597+0000","last_unstale":"2024-05-08T08:25:55.229748+0000","last_undegraded":"2024-05-08T08:25:55.229748+0000","last_fullsized":"2024-05-08T08:25:55.229748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":304,"log_dups_size":0,"ondisk_log_size":304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:21.172919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080728699999999996,"stat_sum":{"num_bytes":46470224,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":15,"num_read":4721,"num_read_kb":20189,"num_write":365,"num_write_kb":14492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"62'201","reported_seq":1320,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:25:53.102630+0000","last_change":"2024-05-08T08:25:48.159653+0000","last_active":"2024-05-08T08:25:53.102630+0000","last_peered":"2024-05-08T08:25:53.102630+0000","last_clean":"2024-05-08T08:25:53.102630+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:25:53.102630+0000","last_undegraded":"2024-05-08T08:25:53.102630+0000","last_fullsized":"2024-05-08T08:25:53.102630+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":201,"log_dups_size":0,"ondisk_log_size":201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:55:25.366740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040135600000000001,"stat_sum":{"num_bytes":20345693,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":11,"num_read":1583,"num_read_kb":18039,"num_write":205,"num_write_kb":9010,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"63'266","reported_seq":3024,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.197946+0000","last_change":"2024-05-08T08:25:06.610175+0000","last_active":"2024-05-08T08:25:55.197946+0000","last_peered":"2024-05-08T08:25:55.197946+0000","last_clean":"2024-05-08T08:25:55.197946+0000","last_became_active":"2024-05-08T08:25:04.609185+0000","last_became_peered":"2024-05-08T08:25:04.609185+0000","last_unstale":"2024-05-08T08:25:55.197946+0000","last_undegraded":"2024-05-08T08:25:55.197946+0000","last_fullsized":"2024-05-08T08:25:55.197946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":266,"log_dups_size":0,"ondisk_log_size":266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:05:16.293122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505687,"stat_sum":{"num_bytes":13183769,"num_objects":35,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":3242,"num_read_kb":14379,"num_write":283,"num_write_kb":13870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"64'608","reported_seq":1393,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.209791+0000","last_change":"2024-05-08T08:25:06.628387+0000","last_active":"2024-05-08T08:25:55.209791+0000","last_peered":"2024-05-08T08:25:55.209791+0000","last_clean":"2024-05-08T08:25:55.209791+0000","last_became_active":"2024-05-08T08:25:04.610818+0000","last_became_peered":"2024-05-08T08:25:04.610818+0000","last_unstale":"2024-05-08T08:25:55.209791+0000","last_undegraded":"2024-05-08T08:25:55.209791+0000","last_fullsized":"2024-05-08T08:25:55.209791+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:16:16.844607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069309899999999997,"stat_sum":{"num_bytes":31054524,"num_objects":40,"num_object_clones":4,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":3160,"num_read_kb":6190,"num_write":922,"num_write_kb":7348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"63'225","reported_seq":1142,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.197759+0000","last_change":"2024-05-08T08:25:06.625296+0000","last_active":"2024-05-08T08:25:55.197759+0000","last_peered":"2024-05-08T08:25:55.197759+0000","last_clean":"2024-05-08T08:25:55.197759+0000","last_became_active":"2024-05-08T08:25:04.605228+0000","last_became_peered":"2024-05-08T08:25:04.605228+0000","last_unstale":"2024-05-08T08:25:55.197759+0000","last_undegraded":"2024-05-08T08:25:55.197759+0000","last_fullsized":"2024-05-08T08:25:55.197759+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":225,"log_dups_size":0,"ondisk_log_size":225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:17:28.763753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011464940000000001,"stat_sum":{"num_bytes":11789894,"num_objects":41,"num_object_clones":7,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":16,"num_read":1556,"num_read_kb":12905,"num_write":211,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"63'274","reported_seq":2954,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237376+0000","last_change":"2024-05-08T08:25:06.796215+0000","last_active":"2024-05-08T08:25:55.237376+0000","last_peered":"2024-05-08T08:25:55.237376+0000","last_clean":"2024-05-08T08:25:55.237376+0000","last_became_active":"2024-05-08T08:25:04.606180+0000","last_became_peered":"2024-05-08T08:25:04.606180+0000","last_unstale":"2024-05-08T08:25:55.237376+0000","last_undegraded":"2024-05-08T08:25:55.237376+0000","last_fullsized":"2024-05-08T08:25:55.237376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":274,"log_dups_size":0,"ondisk_log_size":274,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:07.930707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027126400000000001,"stat_sum":{"num_bytes":31173180,"num_objects":38,"num_object_clones":8,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":14,"num_read":3694,"num_read_kb":17491,"num_write":306,"num_write_kb":4778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"64'385","reported_seq":898,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.211832+0000","last_change":"2024-05-08T08:25:06.615415+0000","last_active":"2024-05-08T08:25:55.211832+0000","last_peered":"2024-05-08T08:25:55.211832+0000","last_clean":"2024-05-08T08:25:55.211832+0000","last_became_active":"2024-05-08T08:25:04.614760+0000","last_became_peered":"2024-05-08T08:25:04.614760+0000","last_unstale":"2024-05-08T08:25:55.211832+0000","last_undegraded":"2024-05-08T08:25:55.211832+0000","last_fullsized":"2024-05-08T08:25:55.211832+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":385,"log_dups_size":0,"ondisk_log_size":385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:31.567373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034964000000000002,"stat_sum":{"num_bytes":17194088,"num_objects":40,"num_object_clones":11,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":12,"num_read":1362,"num_read_kb":7013,"num_write":536,"num_write_kb":6697,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.246346+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:25:55.246346+0000","last_peered":"2024-05-08T08:25:55.246346+0000","last_clean":"2024-05-08T08:25:55.246346+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:25:55.246346+0000","last_undegraded":"2024-05-08T08:25:55.246346+0000","last_fullsized":"2024-05-08T08:25:55.246346+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:32.338505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"64'285","reported_seq":7456,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.197840+0000","last_change":"2024-05-08T08:25:06.625323+0000","last_active":"2024-05-08T08:25:55.197840+0000","last_peered":"2024-05-08T08:25:55.197840+0000","last_clean":"2024-05-08T08:25:55.197840+0000","last_became_active":"2024-05-08T08:25:04.606855+0000","last_became_peered":"2024-05-08T08:25:04.606855+0000","last_unstale":"2024-05-08T08:25:55.197840+0000","last_undegraded":"2024-05-08T08:25:55.197840+0000","last_fullsized":"2024-05-08T08:25:55.197840+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:33:11.674034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011708910000000001,"stat_sum":{"num_bytes":17904080,"num_objects":55,"num_object_clones":14,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":20,"num_read":7560,"num_read_kb":9670,"num_write":287,"num_write_kb":9523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.246345+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:25:55.246345+0000","last_peered":"2024-05-08T08:25:55.246345+0000","last_clean":"2024-05-08T08:25:55.246345+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:25:55.246345+0000","last_undegraded":"2024-05-08T08:25:55.246345+0000","last_fullsized":"2024-05-08T08:25:55.246345+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:54.867771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"62'261","reported_seq":3660,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:25:53.097495+0000","last_change":"2024-05-08T08:25:48.161836+0000","last_active":"2024-05-08T08:25:53.097495+0000","last_peered":"2024-05-08T08:25:53.097495+0000","last_clean":"2024-05-08T08:25:53.097495+0000","last_became_active":"2024-05-08T08:25:48.160202+0000","last_became_peered":"2024-05-08T08:25:48.160202+0000","last_unstale":"2024-05-08T08:25:53.097495+0000","last_undegraded":"2024-05-08T08:25:53.097495+0000","last_fullsized":"2024-05-08T08:25:53.097495+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":261,"log_dups_size":0,"ondisk_log_size":261,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:07:11.126146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847699999999999,"stat_sum":{"num_bytes":38328208,"num_objects":41,"num_object_clones":11,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":17,"num_read":4270,"num_read_kb":28475,"num_write":269,"num_write_kb":5390,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":1182817,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.211756+0000","last_change":"2024-05-08T08:24:56.561965+0000","last_active":"2024-05-08T08:25:55.211756+0000","last_peered":"2024-05-08T08:25:55.211756+0000","last_clean":"2024-05-08T08:25:55.211756+0000","last_became_active":"2024-05-08T08:24:56.561594+0000","last_became_peered":"2024-05-08T08:24:56.561594+0000","last_unstale":"2024-05-08T08:25:55.211756+0000","last_undegraded":"2024-05-08T08:25:55.211756+0000","last_fullsized":"2024-05-08T08:25:55.211756+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:43:18.265730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"64'632","reported_seq":4954,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.198406+0000","last_change":"2024-05-08T08:25:06.611208+0000","last_active":"2024-05-08T08:25:55.198406+0000","last_peered":"2024-05-08T08:25:55.198406+0000","last_clean":"2024-05-08T08:25:55.198406+0000","last_became_active":"2024-05-08T08:25:04.608555+0000","last_became_peered":"2024-05-08T08:25:04.608555+0000","last_unstale":"2024-05-08T08:25:55.198406+0000","last_undegraded":"2024-05-08T08:25:55.198406+0000","last_fullsized":"2024-05-08T08:25:55.198406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":632,"log_dups_size":0,"ondisk_log_size":632,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:12:00.742760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195331,"stat_sum":{"num_bytes":45311392,"num_objects":54,"num_object_clones":11,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":17,"num_read":7139,"num_read_kb":59997,"num_write":892,"num_write_kb":32655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":107,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.198171+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:25:55.198171+0000","last_peered":"2024-05-08T08:25:55.198171+0000","last_clean":"2024-05-08T08:25:55.198171+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:25:55.198171+0000","last_undegraded":"2024-05-08T08:25:55.198171+0000","last_fullsized":"2024-05-08T08:25:55.198171+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"64'298","reported_seq":1802,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.223995+0000","last_change":"2024-05-08T08:25:48.159785+0000","last_active":"2024-05-08T08:25:55.223995+0000","last_peered":"2024-05-08T08:25:55.223995+0000","last_clean":"2024-05-08T08:25:55.223995+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:25:55.223995+0000","last_undegraded":"2024-05-08T08:25:55.223995+0000","last_fullsized":"2024-05-08T08:25:55.223995+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:38:10.978584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062277100000000005,"stat_sum":{"num_bytes":36708900,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":17,"num_read":2242,"num_read_kb":35437,"num_write":316,"num_write_kb":17357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"64'804","reported_seq":1845,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.266871+0000","last_change":"2024-05-08T08:25:06.628002+0000","last_active":"2024-05-08T08:25:55.266871+0000","last_peered":"2024-05-08T08:25:55.266871+0000","last_clean":"2024-05-08T08:25:55.266871+0000","last_became_active":"2024-05-08T08:25:04.615077+0000","last_became_peered":"2024-05-08T08:25:04.615077+0000","last_unstale":"2024-05-08T08:25:55.266871+0000","last_undegraded":"2024-05-08T08:25:55.266871+0000","last_fullsized":"2024-05-08T08:25:55.266871+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":804,"log_dups_size":0,"ondisk_log_size":804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:25.271746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057249000000000004,"stat_sum":{"num_bytes":12063466,"num_objects":49,"num_object_clones":12,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":23,"num_read":4609,"num_read_kb":11428,"num_write":1195,"num_write_kb":7756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":263,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237188+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:25:55.237188+0000","last_peered":"2024-05-08T08:25:55.237188+0000","last_clean":"2024-05-08T08:25:55.237188+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:25:55.237188+0000","last_undegraded":"2024-05-08T08:25:55.237188+0000","last_fullsized":"2024-05-08T08:25:55.237188+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":34,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:25:52.223086+0000","last_change":"2024-05-08T08:25:45.225724+0000","last_active":"2024-05-08T08:25:52.223086+0000","last_peered":"2024-05-08T08:25:52.223086+0000","last_clean":"2024-05-08T08:25:52.223086+0000","last_became_active":"2024-05-08T08:25:45.225388+0000","last_became_peered":"2024-05-08T08:25:45.225388+0000","last_unstale":"2024-05-08T08:25:52.223086+0000","last_undegraded":"2024-05-08T08:25:52.223086+0000","last_fullsized":"2024-05-08T08:25:52.223086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:54.551477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"62'224","reported_seq":988,"reported_epoch":62,"state":"active+clean","last_fresh":"2024-05-08T08:25:53.095523+0000","last_change":"2024-05-08T08:25:48.160149+0000","last_active":"2024-05-08T08:25:53.095523+0000","last_peered":"2024-05-08T08:25:53.095523+0000","last_clean":"2024-05-08T08:25:53.095523+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:25:53.095523+0000","last_undegraded":"2024-05-08T08:25:53.095523+0000","last_fullsized":"2024-05-08T08:25:53.095523+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":224,"log_dups_size":0,"ondisk_log_size":224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:55:21.781922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034192400000000001,"stat_sum":{"num_bytes":27831239,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":637,"num_read_kb":12703,"num_write":262,"num_write_kb":10556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237282+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:25:55.237282+0000","last_peered":"2024-05-08T08:25:55.237282+0000","last_clean":"2024-05-08T08:25:55.237282+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:25:55.237282+0000","last_undegraded":"2024-05-08T08:25:55.237282+0000","last_fullsized":"2024-05-08T08:25:55.237282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"63'255","reported_seq":3238,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.211717+0000","last_change":"2024-05-08T08:25:06.615595+0000","last_active":"2024-05-08T08:25:55.211717+0000","last_peered":"2024-05-08T08:25:55.211717+0000","last_clean":"2024-05-08T08:25:55.211717+0000","last_became_active":"2024-05-08T08:25:04.611900+0000","last_became_peered":"2024-05-08T08:25:04.611900+0000","last_unstale":"2024-05-08T08:25:55.211717+0000","last_undegraded":"2024-05-08T08:25:55.211717+0000","last_fullsized":"2024-05-08T08:25:55.211717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":255,"log_dups_size":0,"ondisk_log_size":255,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:38:42.384987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050119499999999996,"stat_sum":{"num_bytes":20415736,"num_objects":33,"num_object_clones":7,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":12,"num_read":2917,"num_read_kb":21189,"num_write":265,"num_write_kb":16003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.197670+0000","last_change":"2024-05-08T08:24:56.680309+0000","last_active":"2024-05-08T08:25:55.197670+0000","last_peered":"2024-05-08T08:25:55.197670+0000","last_clean":"2024-05-08T08:25:55.197670+0000","last_became_active":"2024-05-08T08:24:56.679967+0000","last_became_peered":"2024-05-08T08:24:56.679967+0000","last_unstale":"2024-05-08T08:25:55.197670+0000","last_undegraded":"2024-05-08T08:25:55.197670+0000","last_fullsized":"2024-05-08T08:25:55.197670+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:16:10.956541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"64'197","reported_seq":1160,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.198598+0000","last_change":"2024-05-08T08:25:06.610137+0000","last_active":"2024-05-08T08:25:55.198598+0000","last_peered":"2024-05-08T08:25:55.198598+0000","last_clean":"2024-05-08T08:25:55.198598+0000","last_became_active":"2024-05-08T08:25:04.606801+0000","last_became_peered":"2024-05-08T08:25:04.606801+0000","last_unstale":"2024-05-08T08:25:55.198598+0000","last_undegraded":"2024-05-08T08:25:55.198598+0000","last_fullsized":"2024-05-08T08:25:55.198598+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":197,"log_dups_size":0,"ondisk_log_size":197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:14:41.438420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042592699999999999,"stat_sum":{"num_bytes":62555602,"num_objects":41,"num_object_clones":10,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":19,"num_read":902,"num_read_kb":46926,"num_write":197,"num_write_kb":20491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":108,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237191+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:25:55.237191+0000","last_peered":"2024-05-08T08:25:55.237191+0000","last_clean":"2024-05-08T08:25:55.237191+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:25:55.237191+0000","last_undegraded":"2024-05-08T08:25:55.237191+0000","last_fullsized":"2024-05-08T08:25:55.237191+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"64'256","reported_seq":2218,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.198257+0000","last_change":"2024-05-08T08:25:48.159093+0000","last_active":"2024-05-08T08:25:55.198257+0000","last_peered":"2024-05-08T08:25:55.198257+0000","last_clean":"2024-05-08T08:25:55.198257+0000","last_became_active":"2024-05-08T08:25:48.158198+0000","last_became_peered":"2024-05-08T08:25:48.158198+0000","last_unstale":"2024-05-08T08:25:55.198257+0000","last_undegraded":"2024-05-08T08:25:55.198257+0000","last_fullsized":"2024-05-08T08:25:55.198257+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:27.014623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001097338,"stat_sum":{"num_bytes":24547312,"num_objects":44,"num_object_clones":7,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":15,"num_read":3091,"num_read_kb":21242,"num_write":291,"num_write_kb":17126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"63'256","reported_seq":1143,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.211649+0000","last_change":"2024-05-08T08:25:06.615379+0000","last_active":"2024-05-08T08:25:55.211649+0000","last_peered":"2024-05-08T08:25:55.211649+0000","last_clean":"2024-05-08T08:25:55.211649+0000","last_became_active":"2024-05-08T08:25:04.901435+0000","last_became_peered":"2024-05-08T08:25:04.901435+0000","last_unstale":"2024-05-08T08:25:55.211649+0000","last_undegraded":"2024-05-08T08:25:55.211649+0000","last_fullsized":"2024-05-08T08:25:55.211649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:23.270521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039550799999999998,"stat_sum":{"num_bytes":11864326,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":2114,"num_read_kb":6458,"num_write":249,"num_write_kb":3731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"63'412","reported_seq":954,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.246274+0000","last_change":"2024-05-08T08:25:06.612336+0000","last_active":"2024-05-08T08:25:55.246274+0000","last_peered":"2024-05-08T08:25:55.246274+0000","last_clean":"2024-05-08T08:25:55.246274+0000","last_became_active":"2024-05-08T08:25:04.615227+0000","last_became_peered":"2024-05-08T08:25:04.615227+0000","last_unstale":"2024-05-08T08:25:55.246274+0000","last_undegraded":"2024-05-08T08:25:55.246274+0000","last_fullsized":"2024-05-08T08:25:55.246274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:39:53.134701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034968800000000002,"stat_sum":{"num_bytes":29191848,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":1558,"num_read_kb":16651,"num_write":555,"num_write_kb":7905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"63'206","reported_seq":1367,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.198018+0000","last_change":"2024-05-08T08:25:06.610868+0000","last_active":"2024-05-08T08:25:55.198018+0000","last_peered":"2024-05-08T08:25:55.198018+0000","last_clean":"2024-05-08T08:25:55.198018+0000","last_became_active":"2024-05-08T08:25:04.608456+0000","last_became_peered":"2024-05-08T08:25:04.608456+0000","last_unstale":"2024-05-08T08:25:55.198018+0000","last_undegraded":"2024-05-08T08:25:55.198018+0000","last_fullsized":"2024-05-08T08:25:55.198018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:29.987597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112149,"stat_sum":{"num_bytes":34548933,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":1100,"num_read_kb":14290,"num_write":206,"num_write_kb":8833,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"63'230","reported_seq":1062,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.197690+0000","last_change":"2024-05-08T08:25:06.624428+0000","last_active":"2024-05-08T08:25:55.197690+0000","last_peered":"2024-05-08T08:25:55.197690+0000","last_clean":"2024-05-08T08:25:55.197690+0000","last_became_active":"2024-05-08T08:25:04.606925+0000","last_became_peered":"2024-05-08T08:25:04.606925+0000","last_unstale":"2024-05-08T08:25:55.197690+0000","last_undegraded":"2024-05-08T08:25:55.197690+0000","last_fullsized":"2024-05-08T08:25:55.197690+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":230,"log_dups_size":0,"ondisk_log_size":230,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:15:25.553051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046376499999999999,"stat_sum":{"num_bytes":60164896,"num_objects":23,"num_object_clones":2,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":9,"num_read":772,"num_read_kb":35221,"num_write":306,"num_write_kb":23509,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"63'235","reported_seq":552,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237382+0000","last_change":"2024-05-08T08:25:06.796419+0000","last_active":"2024-05-08T08:25:55.237382+0000","last_peered":"2024-05-08T08:25:55.237382+0000","last_clean":"2024-05-08T08:25:55.237382+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:25:55.237382+0000","last_undegraded":"2024-05-08T08:25:55.237382+0000","last_fullsized":"2024-05-08T08:25:55.237382+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":235,"log_dups_size":0,"ondisk_log_size":235,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:06:40.703000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046517299999999999,"stat_sum":{"num_bytes":11702604,"num_objects":35,"num_object_clones":8,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":11,"num_read":237,"num_read_kb":7956,"num_write":238,"num_write_kb":5516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"64'370","reported_seq":2417,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237441+0000","last_change":"2024-05-08T08:25:06.796635+0000","last_active":"2024-05-08T08:25:55.237441+0000","last_peered":"2024-05-08T08:25:55.237441+0000","last_clean":"2024-05-08T08:25:55.237441+0000","last_became_active":"2024-05-08T08:25:04.615004+0000","last_became_peered":"2024-05-08T08:25:04.615004+0000","last_unstale":"2024-05-08T08:25:55.237441+0000","last_undegraded":"2024-05-08T08:25:55.237441+0000","last_fullsized":"2024-05-08T08:25:55.237441+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:30:28.064834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060116800000000001,"stat_sum":{"num_bytes":45043417,"num_objects":31,"num_object_clones":3,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":13,"num_read":3426,"num_read_kb":37116,"num_write":386,"num_write_kb":23426,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"63'250","reported_seq":922,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237945+0000","last_change":"2024-05-08T08:25:06.796737+0000","last_active":"2024-05-08T08:25:55.237945+0000","last_peered":"2024-05-08T08:25:55.237945+0000","last_clean":"2024-05-08T08:25:55.237945+0000","last_became_active":"2024-05-08T08:25:04.615942+0000","last_became_peered":"2024-05-08T08:25:04.615942+0000","last_unstale":"2024-05-08T08:25:55.237945+0000","last_undegraded":"2024-05-08T08:25:55.237945+0000","last_fullsized":"2024-05-08T08:25:55.237945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":250,"log_dups_size":0,"ondisk_log_size":250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:13.902350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065917600000000003,"stat_sum":{"num_bytes":7118097,"num_objects":33,"num_object_clones":6,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":10,"num_read":953,"num_read_kb":3140,"num_write":292,"num_write_kb":2751,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"64'575","reported_seq":1717,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.246322+0000","last_change":"2024-05-08T08:25:06.612297+0000","last_active":"2024-05-08T08:25:55.246322+0000","last_peered":"2024-05-08T08:25:55.246322+0000","last_clean":"2024-05-08T08:25:55.246322+0000","last_became_active":"2024-05-08T08:25:04.616212+0000","last_became_peered":"2024-05-08T08:25:04.616212+0000","last_unstale":"2024-05-08T08:25:55.246322+0000","last_undegraded":"2024-05-08T08:25:55.246322+0000","last_fullsized":"2024-05-08T08:25:55.246322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":575,"log_dups_size":0,"ondisk_log_size":575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:10:43.750438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031704699999999999,"stat_sum":{"num_bytes":6079740,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":3086,"num_read_kb":6886,"num_write":870,"num_write_kb":3589,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"63'248","reported_seq":1013,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237210+0000","last_change":"2024-05-08T08:25:06.796231+0000","last_active":"2024-05-08T08:25:55.237210+0000","last_peered":"2024-05-08T08:25:55.237210+0000","last_clean":"2024-05-08T08:25:55.237210+0000","last_became_active":"2024-05-08T08:25:04.615031+0000","last_became_peered":"2024-05-08T08:25:04.615031+0000","last_unstale":"2024-05-08T08:25:55.237210+0000","last_undegraded":"2024-05-08T08:25:55.237210+0000","last_fullsized":"2024-05-08T08:25:55.237210+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":248,"log_dups_size":0,"ondisk_log_size":248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:35:01.102696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040064000000000001,"stat_sum":{"num_bytes":25486922,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":14,"num_read":865,"num_read_kb":19243,"num_write":271,"num_write_kb":5860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"63'275","reported_seq":1191,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.209644+0000","last_change":"2024-05-08T08:25:06.628036+0000","last_active":"2024-05-08T08:25:55.209644+0000","last_peered":"2024-05-08T08:25:55.209644+0000","last_clean":"2024-05-08T08:25:55.209644+0000","last_became_active":"2024-05-08T08:25:04.609651+0000","last_became_peered":"2024-05-08T08:25:04.609651+0000","last_unstale":"2024-05-08T08:25:55.209644+0000","last_undegraded":"2024-05-08T08:25:55.209644+0000","last_fullsized":"2024-05-08T08:25:55.209644+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:26:26.581439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050895200000000002,"stat_sum":{"num_bytes":21492906,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":13,"num_read":1363,"num_read_kb":20423,"num_write":341,"num_write_kb":13823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"63'549","reported_seq":3518,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-05-08T08:25:55.237286+0000","last_change":"2024-05-08T08:25:06.796497+0000","last_active":"2024-05-08T08:25:55.237286+0000","last_peered":"2024-05-08T08:25:55.237286+0000","last_clean":"2024-05-08T08:25:55.237286+0000","last_became_active":"2024-05-08T08:25:04.616040+0000","last_became_peered":"2024-05-08T08:25:04.616040+0000","last_unstale":"2024-05-08T08:25:55.237286+0000","last_undegraded":"2024-05-08T08:25:55.237286+0000","last_fullsized":"2024-05-08T08:25:55.237286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":549,"log_dups_size":0,"ondisk_log_size":549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:41.953716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050921500000000004,"stat_sum":{"num_bytes":49415071,"num_objects":53,"num_object_clones":13,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":15,"num_read":4865,"num_read_kb":16262,"num_write":867,"num_write_kb":12851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":929613850,"num_objects":1289,"num_object_clones":265,"num_object_copies":2578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1289,"num_whiteouts":464,"num_read":90743,"num_read_kb":648543,"num_write":15019,"num_write_kb":372121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":14928057,"num_keys_recovered":23,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":51220480,"data_stored":557847026,"data_compressed":18328131,"data_compressed_allocated":43024384,"data_compressed_original":552979954,"omap_allocated":9107,"internal_metadata":0},"log_size":11609,"ondisk_log_size":11609,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47956,"kb_used_data":8460,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323884,"statfs":{"total":96636764160,"available":96587657216,"internally_reserved":0,"allocated":8663040,"data_stored":89731171,"data_compressed":3083713,"data_compressed_allocated":6619136,"data_compressed_original":88598060,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38136,"kb_used_data":2780,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333704,"statfs":{"total":96636764160,"available":96597712896,"internally_reserved":0,"allocated":2846720,"data_stored":22440641,"data_compressed":703230,"data_compressed_allocated":1675264,"data_compressed_original":21877760,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48040,"kb_used_data":8540,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94323800,"statfs":{"total":96636764160,"available":96587571200,"internally_reserved":0,"allocated":8744960,"data_stored":88362608,"data_compressed":2683537,"data_compressed_allocated":6868992,"data_compressed_original":87333550,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58284,"kb_used_data":9060,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94313556,"statfs":{"total":96636764160,"available":96577081344,"internally_reserved":0,"allocated":9277440,"data_stored":90424049,"data_compressed":3158054,"data_compressed_allocated":6959104,"data_compressed_original":89143464,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50072,"kb_used_data":10560,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94321768,"statfs":{"total":96636764160,"available":96585490432,"internally_reserved":0,"allocated":10813440,"data_stored":106747130,"data_compressed":3398691,"data_compressed_allocated":8667136,"data_compressed_original":105596408,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33808,"kb_used_data":2628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338032,"statfs":{"total":96636764160,"available":96602144768,"internally_reserved":0,"allocated":2691072,"data_stored":17558698,"data_compressed":799606,"data_compressed_allocated":1490944,"data_compressed_original":16935208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928233987,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34800,"kb_used_data":6116,"kb_used_omap":13,"kb_used_meta":28658,"kb_avail":94337040,"statfs":{"total":96636764160,"available":96601128960,"internally_reserved":0,"allocated":6262784,"data_stored":66392283,"data_compressed":1782128,"data_compressed_allocated":4927488,"data_compressed_original":65695744,"omap_allocated":13550,"internal_metadata":29346578},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,16],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574863,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52516,"kb_used_data":7452,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94319324,"statfs":{"total":96636764160,"available":96582987776,"internally_reserved":0,"allocated":7630848,"data_stored":79918890,"data_compressed":2591502,"data_compressed_allocated":5775360,"data_compressed_original":78932983,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6967296,"data_stored":79642069,"data_compressed":2619618,"data_compressed_allocated":5804032,"data_compressed_original":78961655,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5632000,"data_stored":65553820,"data_compressed":1777185,"data_compressed_allocated":4890624,"data_compressed_original":65111385,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1933312,"data_stored":17181180,"data_compressed":796855,"data_compressed_allocated":1482752,"data_compressed_original":16902440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":10162176,"data_stored":106522752,"data_compressed":3478073,"data_compressed_allocated":8744960,"data_compressed_original":105694712,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8536064,"data_stored":90062741,"data_compressed":3175619,"data_compressed_allocated":6975488,"data_compressed_original":89131176,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8077312,"data_stored":88072232,"data_compressed":2739836,"data_compressed_allocated":6918144,"data_compressed_original":87366318,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2101248,"data_stored":22102441,"data_compressed":703230,"data_compressed_allocated":1675264,"data_compressed_original":21877760,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7811072,"data_stored":88709791,"data_compressed":3037715,"data_compressed_allocated":6533120,"data_compressed_original":87934508,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:25:56.474 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T08:25:56.474 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:25:56.474 INFO:tasks.thrashosds.thrasher:Reweighting osd 2 to 0.5827453217165012 2024-05-08T08:25:56.475 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 0.5827453217165012 2024-05-08T08:25:56.505 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:56.506 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:56.509 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:56.516+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50476 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 161792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:25:56.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6216594539999996, 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018840722000000001, 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50476 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915327+0000", 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915329+0000", 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915336+0000", 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4880000000000001e-06 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915342+0000", 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.933988+0000", 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018646593 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934072+0000", 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3626000000000002e-05 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934167+0000", 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5087999999999995e-05 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50477 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 171008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6216369820000001, 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019055283999999999, 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50477 2024-05-08T08:25:56.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915351+0000", 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.311e-06 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915363+0000", 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2031000000000001e-05 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915369+0000", 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6930000000000004e-06 2024-05-08T08:25:56.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934223+0000", 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018853741 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934307+0000", 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4154999999999996e-05 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934404+0000", 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6973999999999997e-05 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50478 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 180224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6216083619999999, 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019257972000000002, 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50478 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915378+0000", 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915387+0000", 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999993e-06 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915396+0000", 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2499999999999995e-06 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915401+0000", 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.259e-06 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934434+0000", 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019032970999999999 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934535+0000", 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100479 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934635+0000", 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100445 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50479 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 189440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6215761029999998, 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019428513000000001, 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50479 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915413+0000", 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915419+0000", 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7320000000000001e-06 2024-05-08T08:25:56.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915424+0000", 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934664+0000", 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019239903999999999 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934740+0000", 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5833000000000004e-05 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934838+0000", 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7832999999999997e-05 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50480 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 198656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6215541919999996, 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019651075, 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50480 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915432+0000", 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915434+0000", 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.221e-06 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915440+0000", 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4770000000000002e-06 2024-05-08T08:25:56.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915444+0000", 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8330000000000001e-06 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934880+0000", 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019435555 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934984+0000", 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010403099999999999 2024-05-08T08:25:56.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935083+0000", 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8566000000000005e-05 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50481 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6215342540000002, 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019849193000000001, 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50481 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:25:56.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915452+0000", 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915454+0000", 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3719999999999999e-06 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915461+0000", 2024-05-08T08:25:56.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6290000000000004e-06 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915466+0000", 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935126+0000", 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019660074 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935221+0000", 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4598000000000005e-05 2024-05-08T08:25:56.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935301+0000", 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.996e-05 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50482 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.621512675, 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020069704000000001, 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.556 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50482 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915476+0000", 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915482+0000", 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4030000000000004e-06 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915487+0000", 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9219999999999996e-06 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935343+0000", 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019856492999999999 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935433+0000", 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9485000000000001e-05 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935543+0000", 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109832 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.560 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50483 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 226304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6214923500000005, 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020296027000000001, 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50483 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915494+0000", 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915496+0000", 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915502+0000", 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7939999999999999e-06 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915507+0000", 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935582+0000", 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020075790999999999 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935672+0000", 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0074999999999994e-05 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935789+0000", 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011681299999999999 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50484 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6214724910000005, 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020571613999999998, 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50484 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915514+0000", 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915516+0000", 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2809999999999998e-06 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915649+0000", 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000132776 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915668+0000", 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9253e-05 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935846+0000", 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020177820999999999 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935950+0000", 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104478 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936085+0000", 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000134618 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50485 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6213058050000004, 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020672174000000001, 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50485 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4700000000000001e-07 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915684+0000", 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5820000000000002e-06 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915692+0000", 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1829999999999994e-06 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915706+0000", 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3747999999999999e-05 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936120+0000", 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020414472999999999 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936205+0000", 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4561000000000006e-05 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936352+0000", 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014698000000000001 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50486 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6212699270000002, 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020885167, 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50486 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:25:56.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915724+0000", 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4200000000000001e-06 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915734+0000", 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0796e-05 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915748+0000", 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3318e-05 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936388+0000", 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020640642000000001 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936491+0000", 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010300400000000001 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936601+0000", 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109523 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.576 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50737 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 208896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6133170190000001, 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018673419, 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50737 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923672+0000", 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.252e-06 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923679+0000", 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923737+0000", 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8035000000000003e-05 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942158+0000", 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018421334000000001 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942240+0000", 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1409000000000006e-05 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942342+0000", 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102252 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50738 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6132336709999997, 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018832728, 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50738 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923753+0000", 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-07 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923756+0000", 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4510000000000001e-06 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923766+0000", 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.049e-05 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923773+0000", 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942387+0000", 2024-05-08T08:25:56.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018614117999999999 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942475+0000", 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7874999999999995e-05 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942585+0000", 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011014300000000001 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50739 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6131687069999998, 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990561999999999, 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50739 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923828+0000", 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0988999999999999e-05 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923837+0000", 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6889999999999993e-06 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923851+0000", 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3528e-05 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942618+0000", 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018766998999999999 2024-05-08T08:25:56.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942704+0000", 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5995000000000006e-05 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942807+0000", 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103837 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50740 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6131253230000002, 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019168166, 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50740 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923861+0000", 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923863+0000", 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-05-08T08:25:56.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923871+0000", 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0569999999999993e-06 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923877+0000", 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942837+0000", 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018959999000000002 2024-05-08T08:25:56.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942917+0000", 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0519000000000006e-05 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943028+0000", 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111402 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.592 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50741 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6131014749999997, 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019309865999999998, 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50741 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923891+0000", 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923894+0000", 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1829999999999998e-06 2024-05-08T08:25:56.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923902+0000", 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0910000000000004e-06 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923907+0000", 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6729999999999997e-06 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943059+0000", 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019151695 2024-05-08T08:25:56.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943123+0000", 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4098999999999997e-05 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943194+0000", 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0970999999999995e-05 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50742 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.6130705509999999, 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019467485, 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50742 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923932+0000", 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6268000000000001e-05 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923939+0000", 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7270000000000007e-06 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923945+0000", 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5419999999999996e-06 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943232+0000", 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019287316999999998 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943315+0000", 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2854000000000003e-05 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943383+0000", 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7427000000000003e-05 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3767315619999998, 2024-05-08T08:25:56.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3767104620000001, 2024-05-08T08:25:56.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:25:56.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:25:56.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.3766898039999997, 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:56.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:25:56.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:25:56.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:56.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:25:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:25:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:56.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:56.613 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:56.614 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:25:56.615 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:56.622+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:56.715 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:56.723+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:56.732 INFO:teuthology.orchestra.run.smithi031.stdout:4554 read 0x30a57cf thru 0x30b1b99 (0xc3cb bytes) 2024-05-08T08:25:56.736 INFO:teuthology.orchestra.run.smithi031.stdout:4555 read 0x9577a8d thru 0x957d8f1 (0x5e65 bytes) 2024-05-08T08:25:56.736 INFO:teuthology.orchestra.run.smithi031.stdout:4556 read 0x464294e thru 0x464fe68 (0xd51b bytes) 2024-05-08T08:25:56.736 INFO:teuthology.orchestra.run.smithi031.stdout:4557 write 0x27a2613 thru 0x27b03f5 (0xdde3 bytes) 2024-05-08T08:25:56.747 INFO:teuthology.orchestra.run.smithi031.stdout:4558 write 0xd3ca141 thru 0xd3d7382 (0xd242 bytes) 2024-05-08T08:25:56.762 INFO:teuthology.orchestra.run.smithi031.stdout:4559 trunc from 0xdb65d22 to 0x3cfc630 2024-05-08T08:25:56.780 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:56.780 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:56.780 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:56.792 INFO:teuthology.orchestra.run.smithi031.stdout:4560 writesame 0x8a835ac thru 0x8a861b7 data_size 0x2(0x2c0c bytes) 2024-05-08T08:25:56.792 INFO:teuthology.orchestra.run.smithi031.stdout:4561 read 0x7dece0a thru 0x7df5b62 (0x8d59 bytes) 2024-05-08T08:25:56.792 INFO:teuthology.orchestra.run.smithi031.stdout:4562 punch from 0x661ec20 to 0x6621873, (0x2c53 bytes) 2024-05-08T08:25:56.792 INFO:teuthology.orchestra.run.smithi031.stdout:4564 punch from 0x326a712 to 0x3277bfc, (0xd4ea bytes) 2024-05-08T08:25:56.792 INFO:teuthology.orchestra.run.smithi031.stdout:4565 write 0x6061f06 thru 0x606c40e (0xa509 bytes) 2024-05-08T08:25:56.795 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:25:56.796 INFO:teuthology.orchestra.run.smithi031.stdout:4568 read 0x27ae387 thru 0x27bd571 (0xf1eb bytes) 2024-05-08T08:25:56.804 INFO:teuthology.orchestra.run.smithi031.stdout:4569 write 0xa05788f thru 0xa06244d (0xabbf bytes) 2024-05-08T08:25:56.816 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:56.824+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:56.819 INFO:teuthology.orchestra.run.smithi031.stdout:4570 trunc from 0xa06244e to 0x3957c81 2024-05-08T08:25:56.839 INFO:teuthology.orchestra.run.smithi031.stdout:4571 punch from 0x2286a79 to 0x22896f1, (0x2c78 bytes) 2024-05-08T08:25:56.853 INFO:teuthology.orchestra.run.smithi031.stdout:4572 write 0x9d942f1 thru 0x9d9de76 (0x9b86 bytes) 2024-05-08T08:25:56.855 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:56.855 INFO:teuthology.orchestra.run.smithi031.stdout:4574 read 0x295f037 thru 0x29673d5 (0x839f bytes) 2024-05-08T08:25:56.855 INFO:teuthology.orchestra.run.smithi031.stdout:4575 punch from 0x6b4dd7b to 0x6b58c81, (0xaf06 bytes) 2024-05-08T08:25:56.865 INFO:teuthology.orchestra.run.smithi031.stdout:4576 trunc from 0x9d9de77 to 0x13e3f37 2024-05-08T08:25:56.908 INFO:teuthology.orchestra.run.smithi031.stdout:4577 writesame 0x9ff9c0d thru 0xa002878 data_size 0x2c(0x8c6c bytes) 2024-05-08T08:25:56.909 INFO:teuthology.orchestra.run.smithi031.stdout:4578 punch from 0x942a446 to 0x942c38b, (0x1f45 bytes) 2024-05-08T08:25:56.912 INFO:teuthology.orchestra.run.smithi031.stdout:4579 trunc from 0xa002879 to 0xd776d80 2024-05-08T08:25:56.917 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:56.925+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:56.918 INFO:teuthology.orchestra.run.smithi031.stdout:4584 write 0xc07cf4 thru 0xc0b334 (0x3641 bytes) 2024-05-08T08:25:56.926 INFO:teuthology.orchestra.run.smithi031.stdout:4585 punch from 0x33735a to 0x33890d, (0x15b3 bytes) 2024-05-08T08:25:56.926 INFO:teuthology.orchestra.run.smithi031.stdout:4586 write 0xa3059cb thru 0xa3100a1 (0xa6d7 bytes) 2024-05-08T08:25:56.935 INFO:teuthology.orchestra.run.smithi031.stdout:4587 read 0x4a60fc3 thru 0x4a68998 (0x79d6 bytes) 2024-05-08T08:25:56.941 INFO:teuthology.orchestra.run.smithi031.stdout:4588 trunc from 0xd776d80 to 0x804acab 2024-05-08T08:25:56.945 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:56.945 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:56.945 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:56.955 INFO:teuthology.orchestra.run.smithi031.stdout:4590 writesame 0x167c230 thru 0x168aa45 data_size 0x3d(0xe816 bytes) 2024-05-08T08:25:56.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:25:56.963 INFO:teuthology.orchestra.run.smithi031.stdout:4591 trunc from 0x804acab to 0x8b94a10 2024-05-08T08:25:56.981 INFO:teuthology.orchestra.run.smithi031.stdout:4592 writesame 0xd902147 thru 0xd90843d data_size 0xf(0x62f7 bytes) 2024-05-08T08:25:56.981 INFO:teuthology.orchestra.run.smithi031.stdout:4593 writesame 0x7e0292c thru 0x7e11349 data_size 0x15(0xea1e bytes) 2024-05-08T08:25:56.989 INFO:teuthology.orchestra.run.smithi031.stdout:4595 trunc from 0xd90843e to 0xad82bee 2024-05-08T08:25:56.997 INFO:teuthology.orchestra.run.smithi031.stdout:4596 writesame 0xa8f72bf thru 0xa8f9304 data_size 0x36(0x2046 bytes) 2024-05-08T08:25:57.002 INFO:teuthology.orchestra.run.smithi031.stdout:4597 punch from 0x26d5d7a to 0x26d70ac, (0x1332 bytes) 2024-05-08T08:25:57.002 INFO:teuthology.orchestra.run.smithi031.stdout:4598 punch from 0x815c00a to 0x8165b1e, (0x9b14 bytes) 2024-05-08T08:25:57.002 INFO:teuthology.orchestra.run.smithi031.stdout:4599 read 0x59978bd thru 0x59a3199 (0xb8dd bytes) 2024-05-08T08:25:57.002 INFO:teuthology.orchestra.run.smithi031.stdout:4600 clone 27 order 23 su 32768 sc 4 2024-05-08T08:25:57.018 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:57.026+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:57.079 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:57.079 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:57.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:6562 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.154855372999997, 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024048491000000002, 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 6562 2024-05-08T08:25:57.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931269+0000", 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931270+0000", 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.19e-06 2024-05-08T08:25:57.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931276+0000", 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4790000000000004e-06 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931286+0000", 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.044e-05 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931292+0000", 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1500000000000004e-06 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931346+0000", 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4274999999999998e-05 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.931398+0000", 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1999999999999997e-05 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.935504+0000", 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041055170000000004 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955221+0000", 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019717443000000001 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955234+0000", 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2191999999999999e-05 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955271+0000", 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7008000000000002e-05 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:14.955317+0000", 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6796999999999999e-05 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7004 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7077888~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897905524999999, 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016625803000000001, 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7004 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188219+0000", 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7800000000000002e-07 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188222+0000", 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7219999999999999e-06 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188229+0000", 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6359999999999997e-06 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188234+0000", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.502e-06 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.203959+0000", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015725330999999999 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204038+0000", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8379000000000003e-05 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204845+0000", 2024-05-08T08:25:57.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00080675500000000004 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7020 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 524288~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897816325999997, 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017204636999999998, 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7020 2024-05-08T08:25:57.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188308+0000", 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:25:57.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188311+0000", 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2409999999999998e-06 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188317+0000", 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7959999999999998e-06 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188322+0000", 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0150000000000003e-06 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204639+0000", 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016316714 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.204730+0000", 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0771000000000002e-05 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205513+0000", 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00078277700000000002 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.092 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7021 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3145728~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897795348999999, 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017850441000000002, 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.093 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7021 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188329+0000", 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6199999999999999e-07 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188331+0000", 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1069999999999999e-06 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188338+0000", 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4060000000000002e-06 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188343+0000", 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4149999999999998e-06 2024-05-08T08:25:57.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205565+0000", 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017221841000000002 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.205665+0000", 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9530000000000004e-05 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206179+0000", 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051477999999999995 2024-05-08T08:25:57.096 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7022 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 5767168~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897774228999999, 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018456931999999999, 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7022 2024-05-08T08:25:57.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188350+0000", 2024-05-08T08:25:57.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188352+0000", 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188358+0000", 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9859999999999999e-06 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188363+0000", 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6460000000000001e-06 2024-05-08T08:25:57.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206229+0000", 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017865941 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206327+0000", 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7888999999999997e-05 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206807+0000", 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048019000000000002 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7038 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 786432~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897668324999998, 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019193815999999999, 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7038 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188456+0000", 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188457+0000", 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.61e-07 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188461+0000", 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3200000000000001e-06 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188469+0000", 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5549999999999997e-06 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188477+0000", 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3559999999999993e-06 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206829+0000", 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018352456 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.206928+0000", 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8981999999999995e-05 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207650+0000", 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00072148600000000002 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7039 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 3407872~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897635584, 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491153000000001, 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7039 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188489+0000", 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188492+0000", 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9799999999999998e-06 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188499+0000", 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7469999999999999e-06 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188505+0000", 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5990000000000001e-06 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207495+0000", 2024-05-08T08:25:57.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990683000000001 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207577+0000", 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.174e-05 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.207980+0000", 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00040296899999999998 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7040 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 6029312~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897612295000002, 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020064733000000001, 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7040 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:57.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188512+0000", 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188516+0000", 2024-05-08T08:25:57.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3299999999999999e-06 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188522+0000", 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3829999999999997e-06 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188527+0000", 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1080000000000001e-06 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208019+0000", 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019491871000000001 2024-05-08T08:25:57.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208118+0000", 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8855000000000001e-05 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208577+0000", 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000458784 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7056 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2097152~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.112 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897531471999997, 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020583127999999999, 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7056 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:57.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188593+0000", 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.77e-07 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188595+0000", 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0679999999999999e-06 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188603+0000", 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999997e-06 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188609+0000", 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5439999999999998e-06 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208623+0000", 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0200141 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.208713+0000", 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9806e-05 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209176+0000", 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00046324300000000001 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7057 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4718592~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897478354999997, 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021149303000000001, 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7057 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188646+0000", 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188647+0000", 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3399999999999999e-07 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188650+0000", 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7100000000000001e-06 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188657+0000", 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.204e-06 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188665+0000", 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4410000000000004e-06 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209211+0000", 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020546426999999999 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209299+0000", 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8139000000000004e-05 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209795+0000", 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049584800000000001 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7058 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7340032~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897448042000001, 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021706842, 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7058 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:57.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188676+0000", 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188677+0000", 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8799999999999998e-07 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188681+0000", 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:57.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188688+0000", 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.494e-06 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188693+0000", 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2349999999999999e-06 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209838+0000", 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021144677000000001 2024-05-08T08:25:57.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.209901+0000", 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3322000000000001e-05 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210383+0000", 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00048203800000000001 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7073 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 2359296~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897352533000003, 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022130316000000001, 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7073 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188772+0000", 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188775+0000", 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3149999999999999e-06 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188782+0000", 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8630000000000008e-06 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188787+0000", 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0220000000000004e-06 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210421+0000", 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.021633454999999999 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210481+0000", 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9911000000000003e-05 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210902+0000", 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042136000000000002 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7074 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 4980736~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897330236999998, 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022761245999999999, 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7074 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.129 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188794+0000", 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188795+0000", 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188797+0000", 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.17e-06 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188803+0000", 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8969999999999996e-06 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188807+0000", 2024-05-08T08:25:57.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7820000000000001e-06 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.210951+0000", 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022143843 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211043+0000", 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2044000000000002e-05 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211555+0000", 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00051209900000000002 2024-05-08T08:25:57.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7075 3.12 3:4f879886:::rbd_data.119152d8614e.0000000000000001:7 [sparse-read 7602176~262144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.897310310000002, 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.025123269, 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7075 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188814+0000", 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.01e-07 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188817+0000", 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3e-06 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188823+0000", 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8939999999999998e-06 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.188828+0000", 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.938e-06 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211585+0000", 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022757553999999999 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.211640+0000", 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5000000000000002e-05 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213937+0000", 2024-05-08T08:25:57.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002297182 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:7090 3.12 3:4c3c5acc:::rbd_data.119120fdbc10.0000000000000001:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 1572864~19660800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 40.887836186000001, 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.017096892999999998, 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 7090 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198288+0000", 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198290+0000", 2024-05-08T08:25:57.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.666e-06 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198297+0000", 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1829999999999997e-06 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198308+0000", 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0641e-05 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.198314+0000", 2024-05-08T08:25:57.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7850000000000003e-06 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.213971+0000", 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015656415 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214141+0000", 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00016982999999999999 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 7", 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.214349+0000", 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00020874199999999999 2024-05-08T08:25:57.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215032+0000", 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00068226599999999995 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215223+0000", 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000191263 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215286+0000", 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2940000000000004e-05 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:16.215385+0000", 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9161999999999997e-05 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10928 3.6 3:66234790:::rbd_data.11917ede465c.0000000000000000:9 [read 4096000~32768] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.947126408999999, 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016562112, 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10928 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:57.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.138998+0000", 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139012+0000", 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3908e-05 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139020+0000", 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.96e-06 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.139026+0000", 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5219999999999998e-06 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155334+0000", 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016308699999999999 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155417+0000", 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2229000000000002e-05 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.155560+0000", 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014340300000000001 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:10943 3.6 3:62dc9057:::rbd_data.119177d1b31a.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc b=[b] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:57.146 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.945364423000001, 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018993326000000001, 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 10943 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140760+0000", 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140761+0000", 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9899999999999999e-07 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140765+0000", 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2289999999999997e-06 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140775+0000", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.9690000000000006e-06 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.140782+0000", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6320000000000002e-06 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156122+0000", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015340006999999999 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.156209+0000", 2024-05-08T08:25:57.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7249000000000004e-05 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159642+0000", 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0034333430000000002 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159700+0000", 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8078999999999997e-05 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.159753+0000", 2024-05-08T08:25:57.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3019000000000002e-05 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:13022 3.17 3:ee16c3de:::rbd_data.1191c87c6f01.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,zero 20828828~4380244] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e35)", 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.552189966, 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020689712999999998, 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 13022 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533934+0000", 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.533938+0000", 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9879999999999998e-06 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534933+0000", 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00099514400000000011 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534955+0000", 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1225000000000001e-05 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.534965+0000", 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0597999999999999e-05 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535030+0000", 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5068999999999997e-05 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535157+0000", 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000126501 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.535401+0000", 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00024435700000000001 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.540584+0000", 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0051824109999999996 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554524+0000", 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.013939890999999999 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554584+0000", 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.05e-05 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:23.554624+0000", 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0028999999999999e-05 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:23289 3.3 e41/18 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head v 41'115, mlcod=39'113)", 2024-05-08T08:25:57.156 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.562435186999998, 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.024290019999999999, 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523689+0000", 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.523690+0000", 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2900000000000002e-07 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545794+0000", 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.022104220000000001 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545810+0000", 2024-05-08T08:25:57.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5420000000000001e-05 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545818+0000", 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0979999999999994e-06 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545894+0000", 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6579e-05 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.545914+0000", 2024-05-08T08:25:57.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9372e-05 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547942+0000", 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0020284220000000002 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.547979+0000", 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6980000000000002e-05 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:25389 3.17 3:ea3e87a1:::rbd_data.11911e381319.0000000000000114:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e45)", 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 22.138131311999999, 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020343165999999999, 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 25389 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947993+0000", 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947994+0000", 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1319999999999999e-06 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.947998+0000", 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2830000000000003e-06 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948009+0000", 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.058e-05 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948014+0000", 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000001e-06 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948057+0000", 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2330000000000003e-05 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.948137+0000", 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9985000000000001e-05 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 5", 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.952162+0000", 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040247959999999998 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:57.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968241+0000", 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.016079751 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968253+0000", 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1296e-05 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968290+0000", 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7011000000000002e-05 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:34.968336+0000", 2024-05-08T08:25:57.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6612e-05 2024-05-08T08:25:57.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.166 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:57.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:25:57.168 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:57.175+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:57.205 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 0.582745 (952e) 2024-05-08T08:25:57.208 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:25:57.270 INFO:teuthology.orchestra.run.smithi031.stdout:leaving image image_client.0-clone26 intact 2024-05-08T08:25:57.270 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:57.278+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:57.335 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:57.335 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:57.335 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:57.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:25:57.389 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:57.396+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:57.406 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-05-08T08:25:57.470 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:57.478+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:57.517 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:57.517 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:57.517 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:57.517 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:57.517 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:57.526 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:25:57.609 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:57.617+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:57.672 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:57.679+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2370 3.4 e24/18 3:237be9c1:::rbd_data.119116cc7dd0.00000000000001eb:head v 24'46, mlcod=23'42)", 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.962419095000001, 2024-05-08T08:25:57.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029578109999999999, 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721287+0000", 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7199999999999997e-07 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721294+0000", 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9800000000000001e-06 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721309+0000", 2024-05-08T08:25:57.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5116999999999999e-05 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9030000000000004e-06 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721383+0000", 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6111000000000005e-05 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721406+0000", 2024-05-08T08:25:57.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2453e-05 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724192+0000", 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027864909999999999 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724244+0000", 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.958205788000001, 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:25:57.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 45.688512250999999, 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.659424485000002, 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:57.695 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.544272788999997, 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:25:57.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:25:57.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.420626367000001, 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:25:57.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:25:57.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:25:57.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.420555220000001, 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:25:57.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:25:57.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.420209490000001, 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:25:57.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:25:57.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.420168200999999, 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:25:57.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:25:57.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:25:57.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20624 3.4 3:21e3449a:::rbd_data.119174045df.0000000000000045:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.419869882, 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029912699999999999, 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0200000000000001e-07 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263841+0000", 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263854+0000", 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2937e-05 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263862+0000", 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6729999999999992e-06 2024-05-08T08:25:57.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263949+0000", 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7406999999999998e-05 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263954+0000", 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266741+0000", 2024-05-08T08:25:57.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027867640000000002 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266827+0000", 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5780999999999997e-05 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21117 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 413696~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.206110952, 2024-05-08T08:25:57.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003161652, 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21117 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477596+0000", 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8050000000000001e-06 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477599+0000", 2024-05-08T08:25:57.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.616e-06 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477604+0000", 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477608+0000", 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480366+0000", 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027578020000000002 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480420+0000", 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3735e-05 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480756+0000", 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033637199999999998 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.206088093999998, 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:25:57.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21738 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 450560~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.059574937000001, 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031591710000000001, 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:57.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21738 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624133+0000", 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624140+0000", 2024-05-08T08:25:57.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624145+0000", 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627084+0000", 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029395129999999999 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627161+0000", 2024-05-08T08:25:57.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7096999999999997e-05 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627290+0000", 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012866399999999999 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.913874264, 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:25:57.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:25:57.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:25:57.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.740 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.186672227999999, 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.741 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:25:57.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:25:57.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:25:57.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 26.967691862999999, 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:57.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:25:57.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.479466498000001, 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.750 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.559202343000001, 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:25:57.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.4111962089999999, 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3203353149999995, 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:57.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:57.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:25:57.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:57.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:25:57.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:25:57.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:57.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:57.767 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:57.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:25:57.810 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:25:57.817+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:25:57.874 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:57.881+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:57.919 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:57.919 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:57.919 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:25:57.919 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:57.928 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:25:57.974 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:57.982+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:58.038 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:25:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:25:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:25:58.038 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:58.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:25:58.077 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:58.083+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:58.179 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:58.186+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:58.198 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:25:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:25:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:25:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:25:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54375 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0449216420000003, 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060789163, 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54375 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160082+0000", 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.35e-06 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160085+0000", 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8430000000000001e-06 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160088+0000", 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.332e-06 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220737+0000", 2024-05-08T08:25:58.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060648805 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220808+0000", 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1247000000000004e-05 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220869+0000", 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1377999999999994e-05 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0449106749999997, 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0449005900000001, 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.044890111, 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:25:58.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:25:58.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:25:58.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:25:58.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0448731850000001, 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:25:58.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:25:58.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0448522650000003, 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.04481395, 2024-05-08T08:25:58.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:25:58.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0447902039999999, 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:25:58.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0447673609999999, 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:25:58.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:25:58.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0447429719999999, 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:25:58.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:25:58.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:25:58.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0447210519999999, 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0446998179999998, 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:25:58.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:25:58.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0446783740000001, 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0436940699999999, 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0436728689999999, 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.260 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0436613990000003, 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:25:58.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0436516390000001, 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.265 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:25:58.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:25:58.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:25:58.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0436418500000002, 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:25:58.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0436313259999999, 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:25:58.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:25:58.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:25:58.277 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0436216460000001, 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:25:58.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:25:58.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:25:58.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:25:58.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:25:58.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:25:58.283 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:25:58.284 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:25:58.285 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:25:58.285 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:58.291+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:58.385 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:25:58.392+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:25:58.400 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:58.400 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:58.400 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:25:58.400 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:58.409 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:25:58.486 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:58.493+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:58.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:25:58.555 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:58.555 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:25:58.555 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:25:58.555 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:25:58.555 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:58.564 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:25:58.607 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:25:58.615+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:25:58.649 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:25:58.650 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:25:58.667 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:25:58.667 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:25:58.672 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:25:58.672 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:25:58.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:25:58.672 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:25:58.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.672 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:25:58.672 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 42.407114755999999, 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:25:58.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:25:58.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14789 3.9 3:90dc9cea:::rbd_data.119174045df.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 33.222016705000001, 2024-05-08T08:25:58.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024649007000000001, 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14789 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456476+0000", 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0279999999999999e-06 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456480+0000", 2024-05-08T08:25:58.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1280000000000001e-06 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456491+0000", 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0436e-05 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456497+0000", 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.64e-06 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456545+0000", 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7432999999999998e-05 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456635+0000", 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9859999999999997e-05 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.460437+0000", 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038026510000000002 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:58.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481041+0000", 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020603652 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481053+0000", 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2301e-05 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481088+0000", 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4669999999999998e-05 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481124+0000", 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6208000000000003e-05 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23289 3.3 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.158874894, 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.028538101999999999, 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:58.684 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23289 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519618+0000", 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3170000000000001e-06 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522813+0000", 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031947130000000001 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522829+0000", 2024-05-08T08:25:58.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6005e-05 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522837+0000", 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8990000000000001e-06 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522902+0000", 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4939999999999998e-05 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523060+0000", 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000158085 2024-05-08T08:25:58.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523351+0000", 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00029110399999999998 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533281+0000", 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099297550000000002 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548076+0000", 2024-05-08T08:25:58.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014795605999999999 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548119+0000", 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2993e-05 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548155+0000", 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5685000000000001e-05 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23675 3.9 3:93098200:::rbd_data.1191ea0f1b1f.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 27.673915509, 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.027687956, 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23675 2024-05-08T08:25:58.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004578+0000", 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.198e-06 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004583+0000", 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5740000000000003e-06 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004593+0000", 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0127e-05 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004599+0000", 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004650+0000", 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999997e-05 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004699+0000", 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9186e-05 2024-05-08T08:25:58.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.009797+0000", 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050987319999999999 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032181+0000", 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022383157000000001 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032193+0000", 2024-05-08T08:25:58.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2561999999999999e-05 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032228+0000", 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4332000000000003e-05 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032264+0000", 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6797e-05 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:32844 3.1a 3:5c5225a1:::rbd_object_map.11915c46bf0c:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.521420669999999, 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0015158669999999, 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 32844 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.696 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157073+0000", 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157079+0000", 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157093+0000", 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4107999999999999e-05 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157102+0000", 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0049999999999993e-06 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157167+0000", 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4935000000000001e-05 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157178+0000", 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158227+0000", 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.001049232 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158355+0000", 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012803000000000001 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158587+0000", 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023214899999999999 2024-05-08T08:25:58.700 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49774 3.3 3:c428cbb1:::rbd_data.11914ad0077c.0000000000000004:1a [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7887697710000001, 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021265240000000001, 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.701 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49774 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0600000000000001e-07 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889724+0000", 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8980000000000001e-06 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889730+0000", 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889735+0000", 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7129999999999996e-06 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910823+0000", 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021087864000000001 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910898+0000", 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5555000000000004e-05 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910987+0000", 2024-05-08T08:25:58.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9196000000000005e-05 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49825 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 187392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7872831729999996, 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02117349, 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.705 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49825 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891211+0000", 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1019999999999999e-06 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891218+0000", 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8569999999999997e-06 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891223+0000", 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.741e-06 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912215+0000", 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020992055999999999 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912288+0000", 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3326999999999998e-05 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912382+0000", 2024-05-08T08:25:58.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3980999999999993e-05 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49826 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 196608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7872630159999998, 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021377759, 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49826 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891231+0000", 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7540000000000001e-06 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891237+0000", 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7030000000000003e-06 2024-05-08T08:25:58.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891242+0000", 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0329999999999999e-06 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912410+0000", 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021168761000000001 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912484+0000", 2024-05-08T08:25:58.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3837999999999993e-05 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912607+0000", 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012231000000000001 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49827 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 205824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:25:58.713 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7872440919999999, 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021638398999999999, 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49827 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891250+0000", 2024-05-08T08:25:58.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9130000000000001e-06 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891256+0000", 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6570000000000002e-06 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891260+0000", 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5959999999999997e-06 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912651+0000", 2024-05-08T08:25:58.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021390180000000002 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912727+0000", 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6477000000000007e-05 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912886+0000", 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000159176 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49828 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 215040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7872244550000005, 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021996597, 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49828 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.718 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891268+0000", 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891270+0000", 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3640000000000002e-06 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891276+0000", 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6019999999999999e-06 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891281+0000", 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913075+0000", 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021794052000000001 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913159+0000", 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4345999999999995e-05 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913264+0000", 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000105071 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.721 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49829 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 224256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7872051429999996, 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022644764000000001, 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.722 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49829 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:25:58.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891289+0000", 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9439999999999999e-06 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891295+0000", 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.896e-06 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891300+0000", 2024-05-08T08:25:58.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8350000000000003e-06 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913295+0000", 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021995627 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913372+0000", 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6736000000000005e-05 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913931+0000", 2024-05-08T08:25:58.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00055940299999999998 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49830 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7871862199999997, 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022779863000000001, 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.726 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49830 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891308+0000", 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9319999999999999e-06 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891313+0000", 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.31e-06 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891318+0000", 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5700000000000003e-06 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913980+0000", 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02266257 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914025+0000", 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4780999999999997e-05 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914085+0000", 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0316999999999999e-05 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49831 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 242688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7871683159999998, 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022998424999999999, 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49831 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891326+0000", 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.951e-06 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891332+0000", 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8200000000000002e-06 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891336+0000", 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6789999999999996e-06 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914125+0000", 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022788283999999999 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914206+0000", 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1599000000000003e-05 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914322+0000", 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00011571299999999999 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49832 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7871501759999999, 2024-05-08T08:25:58.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023170733999999998, 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49832 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891344+0000", 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891350+0000", 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5679999999999999e-06 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891354+0000", 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5750000000000002e-06 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914351+0000", 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022996632999999999 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914423+0000", 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2469e-05 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914512+0000", 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9166999999999995e-05 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49833 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:25:58.738 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7871320959999997, 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023398988999999999, 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49833 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891362+0000", 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8169999999999999e-06 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891367+0000", 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5110000000000003e-06 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891373+0000", 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5890000000000002e-06 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914541+0000", 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023167982 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914614+0000", 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3317000000000003e-05 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914759+0000", 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00014440500000000001 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.742 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50084 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 216064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7795489240000002, 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021113683000000001, 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50084 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:25:58.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898945+0000", 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9520000000000001e-06 2024-05-08T08:25:58.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898951+0000", 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898955+0000", 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8629999999999999e-06 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919894+0000", 2024-05-08T08:25:58.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020938667000000001 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919965+0000", 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0927999999999996e-05 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920057+0000", 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1650999999999994e-05 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50085 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7792908399999998, 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021069139000000001, 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50085 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899202+0000", 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0999999999999997e-07 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899206+0000", 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.861e-06 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899214+0000", 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1289999999999996e-06 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899220+0000", 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5749999999999997e-06 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920083+0000", 2024-05-08T08:25:58.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020862975999999998 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920177+0000", 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3540000000000002e-05 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920270+0000", 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3247999999999999e-05 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50086 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 234496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7792546820000004, 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021223872000000001, 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50086 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.751 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899238+0000", 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1099999999999995e-07 2024-05-08T08:25:58.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899241+0000", 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8100000000000002e-06 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899247+0000", 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4139999999999999e-06 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899252+0000", 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:25:58.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920299+0000", 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021046914999999999 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920373+0000", 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3962999999999996e-05 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920461+0000", 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8419999999999997e-05 2024-05-08T08:25:58.754 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50087 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7792325949999999, 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021404211999999999, 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50087 2024-05-08T08:25:58.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899260+0000", 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0600000000000001e-07 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899262+0000", 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.058e-06 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899267+0000", 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6699999999999999e-06 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899272+0000", 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6920000000000001e-06 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920490+0000", 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021217881000000001 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920561+0000", 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1377000000000005e-05 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920663+0000", 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102128 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50088 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 6.7792123520000001, 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021585928000000001, 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:25:58.759 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50088 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899280+0000", 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899282+0000", 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1330000000000002e-06 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899288+0000", 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6890000000000001e-06 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:25:58.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899293+0000", 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8160000000000001e-06 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920704+0000", 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021411660999999999 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920778+0000", 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3788999999999994e-05 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:25:58.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920865+0000", 2024-05-08T08:25:58.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7437999999999999e-05 2024-05-08T08:25:58.763 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.763 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.763 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.763 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:25:58.763 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:25:58.763 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:25:58.764 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:25:58.764 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:25:58.765 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:25:58.765 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:25:58.765 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:25:58.766 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:58.772+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:58.766 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:25:58.766 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:25:58.767 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:25:58.767 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:25:58.767 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:25:58.767 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:25:58.771 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:25:58.771 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:25:58.865 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:58.872+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:58.911 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:58.912 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:58.920 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:25:58.966 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:58.974+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:59.027 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.027 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.036 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:25:59.077 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:59.085+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:59.168 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:59.175+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:59.182 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.182 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.191 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:25:59.269 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:59.276+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:59.338 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.338 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.347 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:25:59.391 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:59.399+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:59.453 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.453 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.462 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:25:59.505 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:25:59.511+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:25:59.571 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:59.579+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:59.608 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.608 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.617 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:25:59.672 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:59.679+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:59.723 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.723 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.732 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:25:59.773 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:25:59.780+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:25:59.850 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.850 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.860 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:25:59.874 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:25:59.881+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:25:59.952 INFO:teuthology.orchestra.run.smithi031.stdout:4601 punch from 0x3902e0a to 0x390ea24, (0xbc1a bytes) 2024-05-08T08:25:59.958 INFO:teuthology.orchestra.run.smithi031.stdout:4603 trunc from 0xad82bee to 0xadde33d 2024-05-08T08:25:59.966 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:25:59.966 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:25:59.966 INFO:teuthology.orchestra.run.smithi031.stdout:4604 trunc from 0xadde33d to 0xd516044 2024-05-08T08:25:59.973 INFO:teuthology.orchestra.run.smithi031.stdout:4605 read 0x6df7add thru 0x6dfa43b (0x295f bytes) 2024-05-08T08:25:59.974 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:25:59.982+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:25:59.975 INFO:teuthology.orchestra.run.smithi031.stdout:4607 trunc from 0xd516044 to 0xc099fb8 2024-05-08T08:25:59.975 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:25:59.982 INFO:teuthology.orchestra.run.smithi031.stdout:4609 trunc from 0xc099fb8 to 0xd6991dc 2024-05-08T08:25:59.989 INFO:teuthology.orchestra.run.smithi031.stdout:4611 read 0x59aeaa thru 0x5a11ef (0x6346 bytes) 2024-05-08T08:25:59.989 INFO:teuthology.orchestra.run.smithi031.stdout:4612 read 0x2e6ec6e thru 0x2e7a03b (0xb3ce bytes) 2024-05-08T08:25:59.989 INFO:teuthology.orchestra.run.smithi031.stdout:4613 read 0x84815e2 thru 0x84846e6 (0x3105 bytes) 2024-05-08T08:25:59.989 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:59.989 INFO:teuthology.orchestra.run.smithi031.stdout:4615 write 0x219f4ed thru 0x21ac795 (0xd2a9 bytes) 2024-05-08T08:25:59.995 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:25:59.995 INFO:teuthology.orchestra.run.smithi031.stdout:4617 writesame 0x99ea7c3 thru 0x99f8e5a data_size 0x9d(0xe698 bytes) 2024-05-08T08:26:00.001 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.001 INFO:teuthology.orchestra.run.smithi031.stdout:4620 read 0xa69484 thru 0xa6ba0e (0x258b bytes) 2024-05-08T08:26:00.001 INFO:teuthology.orchestra.run.smithi031.stdout:4621 read 0x48cc505 thru 0x48ccf59 (0xa55 bytes) 2024-05-08T08:26:00.001 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.001 INFO:teuthology.orchestra.run.smithi031.stdout:4625 write 0xd5e1c78 thru 0xd5e21b8 (0x541 bytes) 2024-05-08T08:26:00.004 INFO:teuthology.orchestra.run.smithi031.stdout:4626 read 0x9b9e28a thru 0x9ba0e6d (0x2be4 bytes) 2024-05-08T08:26:00.005 INFO:teuthology.orchestra.run.smithi031.stdout:4627 write 0x5409f81 thru 0x540a87e (0x8fe bytes) 2024-05-08T08:26:00.009 INFO:teuthology.orchestra.run.smithi031.stdout:4628 read 0x968fd65 thru 0x9698797 (0x8a33 bytes) 2024-05-08T08:26:00.009 INFO:teuthology.orchestra.run.smithi031.stdout:4629 read 0x4865264 thru 0x486fc37 (0xa9d4 bytes) 2024-05-08T08:26:00.009 INFO:teuthology.orchestra.run.smithi031.stdout:4630 read 0x5762574 thru 0x5771148 (0xebd5 bytes) 2024-05-08T08:26:00.010 INFO:teuthology.orchestra.run.smithi031.stdout:4631 write 0xb029031 thru 0xb030d3f (0x7d0f bytes) 2024-05-08T08:26:00.017 INFO:teuthology.orchestra.run.smithi031.stdout:4634 read 0x7da71cd thru 0x7db4acc (0xd900 bytes) 2024-05-08T08:26:00.028 INFO:teuthology.orchestra.run.smithi031.stdout:4636 trunc from 0xd6991dc to 0x42eb413 2024-05-08T08:26:00.063 INFO:teuthology.orchestra.run.smithi031.stdout:4639 write 0xec16619 thru 0xec25e99 (0xf881 bytes) 2024-05-08T08:26:00.067 INFO:teuthology.orchestra.run.smithi031.stdout:4640 punch from 0xc0476bf to 0xc04f1da, (0x7b1b bytes) 2024-05-08T08:26:00.068 INFO:teuthology.orchestra.run.smithi031.stdout:4641 writesame 0x5862a9d thru 0x5870801 data_size 0x13(0xdd65 bytes) 2024-05-08T08:26:00.070 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.070 INFO:teuthology.orchestra.run.smithi031.stdout:4643 writesame 0x19c1831 thru 0x19c4ddb data_size 0x1(0x35ab bytes) 2024-05-08T08:26:00.075 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:00.083+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:00.095 INFO:teuthology.orchestra.run.smithi031.stdout:4644 read 0x9cdd34a thru 0x9ce3309 (0x5fc0 bytes) 2024-05-08T08:26:00.095 INFO:teuthology.orchestra.run.smithi031.stdout:4645 write 0x93f4e4f thru 0x94017c5 (0xc977 bytes) 2024-05-08T08:26:00.100 INFO:teuthology.orchestra.run.smithi031.stdout:4646 punch from 0x1d4bc28 to 0x1d564f4, (0xa8cc bytes) 2024-05-08T08:26:00.100 INFO:teuthology.orchestra.run.smithi031.stdout:4647 write 0x41db749 thru 0x41ddcdd (0x2595 bytes) 2024-05-08T08:26:00.101 INFO:teuthology.orchestra.run.smithi031.stdout:4648 read 0x432ce84 thru 0x433aeb0 (0xe02d bytes) 2024-05-08T08:26:00.105 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:00.105 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:00.105 INFO:teuthology.orchestra.run.smithi031.stdout:4649 trunc from 0xec25e9a to 0xbee5d42 2024-05-08T08:26:00.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:00.117 INFO:teuthology.orchestra.run.smithi031.stdout:4650 write 0xb610844 thru 0xb61be3b (0xb5f8 bytes) 2024-05-08T08:26:00.126 INFO:teuthology.orchestra.run.smithi031.stdout:4652 trunc from 0xbee5d42 to 0x74909ee 2024-05-08T08:26:00.137 INFO:teuthology.orchestra.run.smithi031.stdout:4653 read 0x56d56fa thru 0x56def27 (0x982e bytes) 2024-05-08T08:26:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:4654 read 0xc91822 thru 0xca0aa3 (0xf282 bytes) 2024-05-08T08:26:00.138 INFO:teuthology.orchestra.run.smithi031.stdout:4655 writesame 0x735e48c thru 0x736296a data_size 0x9(0x44df bytes) 2024-05-08T08:26:00.145 INFO:teuthology.orchestra.run.smithi031.stdout:4656 read 0x4e34923 thru 0x4e3d5e6 (0x8cc4 bytes) 2024-05-08T08:26:00.145 INFO:teuthology.orchestra.run.smithi031.stdout:4657 writesame 0x368c3fd thru 0x369a504 data_size 0x26(0xe108 bytes) 2024-05-08T08:26:00.164 INFO:teuthology.orchestra.run.smithi031.stdout:4658 writesame 0xea723c6 thru 0xea7da15 data_size 0x8(0xb650 bytes) 2024-05-08T08:26:00.164 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.164 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.164 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.165 INFO:teuthology.orchestra.run.smithi031.stdout:4663 write 0x49621c9 thru 0x4966b93 (0x49cb bytes) 2024-05-08T08:26:00.165 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.166 INFO:teuthology.orchestra.run.smithi031.stdout:4666 writesame 0x5644e5d thru 0x5653962 data_size 0x43(0xeb06 bytes) 2024-05-08T08:26:00.171 INFO:teuthology.orchestra.run.smithi031.stdout:4668 punch from 0x8e35dcf to 0x8e40f6e, (0xb19f bytes) 2024-05-08T08:26:00.171 INFO:teuthology.orchestra.run.smithi031.stdout:4669 writesame 0x992b99b thru 0x992d429 data_size 0x1(0x1a8f bytes) 2024-05-08T08:26:00.176 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:00.184+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:00.180 INFO:teuthology.orchestra.run.smithi031.stdout:4671 read 0x4f9df87 thru 0x4fa746f (0x94e9 bytes) 2024-05-08T08:26:00.181 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.181 INFO:teuthology.orchestra.run.smithi031.stdout:4675 writesame 0xbb8514a thru 0xbb8ced9 data_size 0x29(0x7d90 bytes) 2024-05-08T08:26:00.188 INFO:teuthology.orchestra.run.smithi031.stdout:4676 punch from 0xd797810 to 0xd7a4cf4, (0xd4e4 bytes) 2024-05-08T08:26:00.188 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.188 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.188 INFO:teuthology.orchestra.run.smithi031.stdout:4679 read 0xcb34248 thru 0xcb4152b (0xd2e4 bytes) 2024-05-08T08:26:00.188 INFO:teuthology.orchestra.run.smithi031.stdout:4683 write 0x552d347 thru 0x553c4e6 (0xf1a0 bytes) 2024-05-08T08:26:00.190 INFO:teuthology.orchestra.run.smithi031.stdout:4684 write 0xda85a1 thru 0xdb2027 (0x9a87 bytes) 2024-05-08T08:26:00.227 INFO:teuthology.orchestra.run.smithi031.stdout:4685 trunc from 0xea7da16 to 0xe6dbc67 2024-05-08T08:26:00.233 INFO:teuthology.orchestra.run.smithi031.stdout:4687 write 0xdb37e58 thru 0xdb3a7ee (0x2997 bytes) 2024-05-08T08:26:00.237 INFO:teuthology.orchestra.run.smithi031.stdout:4688 write 0x2479882 thru 0x2483828 (0x9fa7 bytes) 2024-05-08T08:26:00.239 INFO:teuthology.orchestra.run.smithi031.stdout:4689 punch from 0x10f4ce6 to 0x10fbd68, (0x7082 bytes) 2024-05-08T08:26:00.239 INFO:teuthology.orchestra.run.smithi031.stdout:4691 punch from 0x82ffa9a to 0x8308469, (0x89cf bytes) 2024-05-08T08:26:00.239 INFO:teuthology.orchestra.run.smithi031.stdout:4693 read 0xa4c9fd8 thru 0xa4d34ab (0x94d4 bytes) 2024-05-08T08:26:00.240 INFO:teuthology.orchestra.run.smithi031.stdout:4694 write 0x8c68568 thru 0x8c6fe77 (0x7910 bytes) 2024-05-08T08:26:00.241 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.241 INFO:teuthology.orchestra.run.smithi031.stdout:4696 write 0xad09cc3 thru 0xad158fd (0xbc3b bytes) 2024-05-08T08:26:00.248 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:00.248 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:00.251 INFO:teuthology.orchestra.run.smithi031.stdout:4698 trunc from 0xe6dbc67 to 0x8b41d98 2024-05-08T08:26:00.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:00.274 INFO:teuthology.orchestra.run.smithi031.stdout:4699 write 0xb8d48de thru 0xb8daf86 (0x66a9 bytes) 2024-05-08T08:26:00.278 INFO:teuthology.orchestra.run.smithi031.stdout:4700 read 0x8890a8 thru 0x889d72 (0xccb bytes) 2024-05-08T08:26:00.279 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:00.286+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:00.279 INFO:teuthology.orchestra.run.smithi031.stdout:4702 read 0x1f3b74d thru 0x1f4b2e3 (0xfb97 bytes) 2024-05-08T08:26:00.279 INFO:teuthology.orchestra.run.smithi031.stdout:4703 punch from 0xb37fdc6 to 0xb38c0f9, (0xc333 bytes) 2024-05-08T08:26:00.279 INFO:teuthology.orchestra.run.smithi031.stdout:4704 write 0xb22a7bd thru 0xb22dbd2 (0x3416 bytes) 2024-05-08T08:26:00.282 INFO:teuthology.orchestra.run.smithi031.stdout:4705 write 0xa7e81dd thru 0xa7f20e6 (0x9f0a bytes) 2024-05-08T08:26:00.284 INFO:teuthology.orchestra.run.smithi031.stdout:4706 punch from 0x5f896ae to 0x5f93049, (0x999b bytes) 2024-05-08T08:26:00.286 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:26:00.297 INFO:teuthology.orchestra.run.smithi031.stdout:4707 trunc from 0xb8daf87 to 0x433c4 2024-05-08T08:26:00.322 INFO:teuthology.orchestra.run.smithi031.stdout:4708 read 0x336b8 thru 0x408f9 (0xd242 bytes) 2024-05-08T08:26:00.323 INFO:teuthology.orchestra.run.smithi031.stdout:4709 punch from 0x3bf5e to 0x433c4, (0x7466 bytes) 2024-05-08T08:26:00.323 INFO:teuthology.orchestra.run.smithi031.stdout:4710 read 0x17032 thru 0x1b9c1 (0x4990 bytes) 2024-05-08T08:26:00.324 INFO:teuthology.orchestra.run.smithi031.stdout:4711 read 0x2c839 thru 0x381d8 (0xb9a0 bytes) 2024-05-08T08:26:00.324 INFO:teuthology.orchestra.run.smithi031.stdout:4712 punch from 0x35738 to 0x403d3, (0xac9b bytes) 2024-05-08T08:26:00.324 INFO:teuthology.orchestra.run.smithi031.stdout:4713 read 0x15fc3 thru 0x215f6 (0xb634 bytes) 2024-05-08T08:26:00.337 INFO:teuthology.orchestra.run.smithi031.stdout:4714 trunc from 0x433c4 to 0xb1a28fb 2024-05-08T08:26:00.343 INFO:teuthology.orchestra.run.smithi031.stdout:4715 write 0x2a710eb thru 0x2a76d46 (0x5c5c bytes) 2024-05-08T08:26:00.347 INFO:teuthology.orchestra.run.smithi031.stdout:4716 punch from 0x56f9430 to 0x57015b2, (0x8182 bytes) 2024-05-08T08:26:00.347 INFO:teuthology.orchestra.run.smithi031.stdout:4717 writesame 0x5cf530e thru 0x5cf93bf data_size 0x31(0x40b2 bytes) 2024-05-08T08:26:00.350 INFO:teuthology.orchestra.run.smithi031.stdout:4718 write 0x2c22ca8 thru 0x2c30476 (0xd7cf bytes) 2024-05-08T08:26:00.355 INFO:teuthology.orchestra.run.smithi031.stdout:4719 read 0x69f5e50 thru 0x6a04c47 (0xedf8 bytes) 2024-05-08T08:26:00.355 INFO:teuthology.orchestra.run.smithi031.stdout:4720 write 0x14a5ba8 thru 0x14b573e (0xfb97 bytes) 2024-05-08T08:26:00.363 INFO:teuthology.orchestra.run.smithi031.stdout:4721 trunc from 0xb1a28fb to 0x5b276c1 2024-05-08T08:26:00.370 INFO:teuthology.orchestra.run.smithi031.stdout:4722 read 0x18a6ea7 thru 0x18a7266 (0x3c0 bytes) 2024-05-08T08:26:00.370 INFO:teuthology.orchestra.run.smithi031.stdout:4723 read 0x135281c thru 0x1353b77 (0x135c bytes) 2024-05-08T08:26:00.371 INFO:teuthology.orchestra.run.smithi031.stdout:4724 read 0x22c555e thru 0x22cbb96 (0x6639 bytes) 2024-05-08T08:26:00.374 INFO:teuthology.orchestra.run.smithi031.stdout:4725 trunc from 0x5b276c1 to 0x80fbb9d 2024-05-08T08:26:00.378 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:00.385+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:00.379 INFO:teuthology.orchestra.run.smithi031.stdout:4726 writesame 0x1d869f8 thru 0x1d88b42 data_size 0x3(0x214b bytes) 2024-05-08T08:26:00.386 INFO:teuthology.orchestra.run.smithi031.stdout:4727 trunc from 0x80fbb9d to 0xbaf4edb 2024-05-08T08:26:00.391 INFO:teuthology.orchestra.run.smithi031.stdout:4730 writesame 0x29b015e thru 0x29bab30 data_size 0x2f(0xa9d3 bytes) 2024-05-08T08:26:00.395 INFO:teuthology.orchestra.run.smithi031.stdout:4731 punch from 0x6bc7c9c to 0x6bcdacf, (0x5e33 bytes) 2024-05-08T08:26:00.395 INFO:teuthology.orchestra.run.smithi031.stdout:4732 write 0x92abf81 thru 0x92b5f8f (0xa00f bytes) 2024-05-08T08:26:00.398 INFO:teuthology.orchestra.run.smithi031.stdout:4734 write 0x62506dc thru 0x6255fb7 (0x58dc bytes) 2024-05-08T08:26:00.405 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:00.405 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:00.408 INFO:teuthology.orchestra.run.smithi031.stdout:4735 write 0xc475258 thru 0xc477a92 (0x283b bytes) 2024-05-08T08:26:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:4739 read 0x68c6d30 thru 0x68d62d8 (0xf5a9 bytes) 2024-05-08T08:26:00.411 INFO:teuthology.orchestra.run.smithi031.stdout:4740 punch from 0x2235211 to 0x22394f6, (0x42e5 bytes) 2024-05-08T08:26:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:4741 punch from 0x32e7402 to 0x32e74be, (0xbc bytes) 2024-05-08T08:26:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:4742 read 0x9529d2c thru 0x9539bb9 (0xfe8e bytes) 2024-05-08T08:26:00.412 INFO:teuthology.orchestra.run.smithi031.stdout:4743 read 0xc66f94 thru 0xc7035a (0x93c7 bytes) 2024-05-08T08:26:00.415 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:00.419 INFO:teuthology.orchestra.run.smithi031.stdout:4744 writesame 0xd161470 thru 0xd16bd79 data_size 0x4d(0xa90a bytes) 2024-05-08T08:26:00.419 INFO:teuthology.orchestra.run.smithi031.stdout:4745 writesame 0x1b9ae44 thru 0x1b9d211 data_size 0x1(0x23ce bytes) 2024-05-08T08:26:00.424 INFO:teuthology.orchestra.run.smithi031.stdout:4746 read 0xa370366 thru 0xa37f193 (0xee2e bytes) 2024-05-08T08:26:00.425 INFO:teuthology.orchestra.run.smithi031.stdout:4747 write 0x4fdccb0 thru 0x4fea12a (0xd47b bytes) 2024-05-08T08:26:00.428 INFO:teuthology.orchestra.run.smithi031.stdout:4748 write 0xbefbf23 thru 0xbf04ae3 (0x8bc1 bytes) 2024-05-08T08:26:00.433 INFO:teuthology.orchestra.run.smithi031.stdout:4749 write 0x622cfdb thru 0x6235d2a (0x8d50 bytes) 2024-05-08T08:26:00.436 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.436 INFO:teuthology.orchestra.run.smithi031.stdout:4751 punch from 0xa144d23 to 0xa145946, (0xc23 bytes) 2024-05-08T08:26:00.436 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.436 INFO:teuthology.orchestra.run.smithi031.stdout:4753 write 0x1b110f3 thru 0x1b1d086 (0xbf94 bytes) 2024-05-08T08:26:00.440 INFO:teuthology.orchestra.run.smithi031.stdout:4754 trunc from 0xd16bd7a to 0xaf602dd 2024-05-08T08:26:00.452 INFO:teuthology.orchestra.run.smithi031.stdout:4755 punch from 0x277d72b to 0x27813e6, (0x3cbb bytes) 2024-05-08T08:26:00.452 INFO:teuthology.orchestra.run.smithi031.stdout:4757 write 0x7abf673 thru 0x7ac5a80 (0x640e bytes) 2024-05-08T08:26:00.461 INFO:teuthology.orchestra.run.smithi031.stdout:4758 trunc from 0xaf602dd to 0xa2f31c8 2024-05-08T08:26:00.471 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.471 INFO:teuthology.orchestra.run.smithi031.stdout:4762 write 0x2d51cdd thru 0x2d51eda (0x1fe bytes) 2024-05-08T08:26:00.473 INFO:teuthology.orchestra.run.smithi031.stdout:4764 write 0x17f8776 thru 0x1807214 (0xea9f bytes) 2024-05-08T08:26:00.479 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:00.486+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:00.480 INFO:teuthology.orchestra.run.smithi031.stdout:4765 read 0x9f8da93 thru 0x9f8fe84 (0x23f2 bytes) 2024-05-08T08:26:00.481 INFO:teuthology.orchestra.run.smithi031.stdout:4766 read 0xea4cb4 thru 0xeb4604 (0xf951 bytes) 2024-05-08T08:26:00.481 INFO:teuthology.orchestra.run.smithi031.stdout:4768 punch from 0x65e439b to 0x65ed916, (0x957b bytes) 2024-05-08T08:26:00.490 INFO:teuthology.orchestra.run.smithi031.stdout:4769 write 0xac663a6 thru 0xac68ed7 (0x2b32 bytes) 2024-05-08T08:26:00.493 INFO:teuthology.orchestra.run.smithi031.stdout:4770 punch from 0x56d9a15 to 0x56ddf8c, (0x4577 bytes) 2024-05-08T08:26:00.493 INFO:teuthology.orchestra.run.smithi031.stdout:4773 write 0x740aaee thru 0x7416978 (0xbe8b bytes) 2024-05-08T08:26:00.497 INFO:teuthology.orchestra.run.smithi031.stdout:4774 write 0x5e102b thru 0x5ecc5e (0xbc34 bytes) 2024-05-08T08:26:00.501 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.502 INFO:teuthology.orchestra.run.smithi031.stdout:4776 punch from 0x1b7bec2 to 0x1b7f770, (0x38ae bytes) 2024-05-08T08:26:00.502 INFO:teuthology.orchestra.run.smithi031.stdout:4777 punch from 0x27698a9 to 0x276bf8a, (0x26e1 bytes) 2024-05-08T08:26:00.504 INFO:teuthology.orchestra.run.smithi031.stdout:4778 trunc from 0xac68ed8 to 0xd0aa268 2024-05-08T08:26:00.512 INFO:teuthology.orchestra.run.smithi031.stdout:4779 trunc from 0xd0aa268 to 0xb67af69 2024-05-08T08:26:00.521 INFO:teuthology.orchestra.run.smithi031.stdout:4780 read 0x5233cb3 thru 0x5241a01 (0xdd4f bytes) 2024-05-08T08:26:00.521 INFO:teuthology.orchestra.run.smithi031.stdout:4781 read 0x99bb575 thru 0x99ca8e2 (0xf36e bytes) 2024-05-08T08:26:00.522 INFO:teuthology.orchestra.run.smithi031.stdout:4783 writesame 0x5b7f9f1 thru 0x5b8b8a2 data_size 0x16(0xbeb2 bytes) 2024-05-08T08:26:00.524 INFO:teuthology.orchestra.run.smithi031.stdout:4784 punch from 0x52e6e7b to 0x52edfe9, (0x716e bytes) 2024-05-08T08:26:00.531 INFO:teuthology.orchestra.run.smithi031.stdout:4785 writesame 0xc287350 thru 0xc28e03f data_size 0x1c(0x6cf0 bytes) 2024-05-08T08:26:00.531 INFO:teuthology.orchestra.run.smithi031.stdout:4786 writesame 0x71778bc thru 0x717f992 data_size 0x1(0x80d7 bytes) 2024-05-08T08:26:00.541 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:00.542 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:00.545 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.546 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.546 INFO:teuthology.orchestra.run.smithi031.stdout:4791 read 0x2e64b2e thru 0x2e6bc86 (0x7159 bytes) 2024-05-08T08:26:00.552 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:00.556 INFO:teuthology.orchestra.run.smithi031.stdout:4792 writesame 0xdec9f04 thru 0xded3b7d data_size 0x1(0x9c7a bytes) 2024-05-08T08:26:00.557 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.570 INFO:teuthology.orchestra.run.smithi031.stdout:4795 trunc from 0xded3b7e to 0x19cbe68 2024-05-08T08:26:00.580 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:00.588+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:00.590 INFO:teuthology.orchestra.run.smithi031.stdout:4796 punch from 0x112c091 to 0x11324f0, (0x645f bytes) 2024-05-08T08:26:00.590 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.593 INFO:teuthology.orchestra.run.smithi031.stdout:4798 trunc from 0x19cbe68 to 0x3cb906f 2024-05-08T08:26:00.603 INFO:teuthology.orchestra.run.smithi031.stdout:4800 trunc from 0x3cb906f to 0x75055ce 2024-05-08T08:26:00.608 INFO:teuthology.orchestra.run.smithi031.stdout:4801 punch from 0xe74bc6 to 0xe7a6c7, (0x5b01 bytes) 2024-05-08T08:26:00.608 INFO:teuthology.orchestra.run.smithi031.stdout:4802 read 0x5d4daf3 thru 0x5d546eb (0x6bf9 bytes) 2024-05-08T08:26:00.608 INFO:teuthology.orchestra.run.smithi031.stdout:4804 write 0x53cb2af thru 0x53d9687 (0xe3d9 bytes) 2024-05-08T08:26:00.617 INFO:teuthology.orchestra.run.smithi031.stdout:4805 trunc from 0x75055ce to 0x78eab8e 2024-05-08T08:26:00.631 INFO:teuthology.orchestra.run.smithi031.stdout:4808 writesame 0xa0066d1 thru 0xa00f94a data_size 0x1(0x927a bytes) 2024-05-08T08:26:00.633 INFO:teuthology.orchestra.run.smithi031.stdout:4809 trunc from 0xa00f94b to 0x8c53e71 2024-05-08T08:26:00.639 INFO:teuthology.orchestra.run.smithi031.stdout:4810 read 0x140d497 thru 0x14161b4 (0x8d1e bytes) 2024-05-08T08:26:00.646 INFO:teuthology.orchestra.run.smithi031.stdout:4813 write 0x9acfc6e thru 0x9ad37da (0x3b6d bytes) 2024-05-08T08:26:00.650 INFO:teuthology.orchestra.run.smithi031.stdout:4815 punch from 0x2d6a6b7 to 0x2d716ec, (0x7035 bytes) 2024-05-08T08:26:00.650 INFO:teuthology.orchestra.run.smithi031.stdout:4818 read 0x4752a8e thru 0x47584e0 (0x5a53 bytes) 2024-05-08T08:26:00.650 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.650 INFO:teuthology.orchestra.run.smithi031.stdout:4820 read 0x5f9ca80 thru 0x5fa4ca6 (0x8227 bytes) 2024-05-08T08:26:00.657 INFO:teuthology.orchestra.run.smithi031.stdout:4822 trunc from 0x9ad37db to 0x373510d 2024-05-08T08:26:00.669 INFO:teuthology.orchestra.run.smithi031.stdout:4825 punch from 0xde1639 to 0xdee3e1, (0xcda8 bytes) 2024-05-08T08:26:00.669 INFO:teuthology.orchestra.run.smithi031.stdout:4827 read 0x18dc762 thru 0x18e7694 (0xaf33 bytes) 2024-05-08T08:26:00.681 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:00.688+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:00.683 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:00.683 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:00.684 INFO:teuthology.orchestra.run.smithi031.stdout:4828 writesame 0xb88c897 thru 0xb89a35f data_size 0x1(0xdac9 bytes) 2024-05-08T08:26:00.684 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.684 INFO:teuthology.orchestra.run.smithi031.stdout:4830 write 0x62dd9f thru 0x6381bf (0xa421 bytes) 2024-05-08T08:26:00.686 INFO:teuthology.orchestra.run.smithi031.stdout:4831 punch from 0x5a4c61d to 0x5a4db95, (0x1578 bytes) 2024-05-08T08:26:00.692 INFO:teuthology.orchestra.run.smithi031.stdout:4832 writesame 0xbe219a4 thru 0xbe29685 data_size 0x2e(0x7ce2 bytes) 2024-05-08T08:26:00.692 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.692 INFO:teuthology.orchestra.run.smithi031.stdout:4836 writesame 0x3c38f4d thru 0x3c3ac1e data_size 0x22(0x1cd2 bytes) 2024-05-08T08:26:00.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:00.695 INFO:teuthology.orchestra.run.smithi031.stdout:4837 punch from 0xbc41c75 to 0xbc455ac, (0x3937 bytes) 2024-05-08T08:26:00.695 INFO:teuthology.orchestra.run.smithi031.stdout:4838 write 0x5228409 thru 0x522ac67 (0x285f bytes) 2024-05-08T08:26:00.698 INFO:teuthology.orchestra.run.smithi031.stdout:4841 read 0x25690ef thru 0x257769b (0xe5ad bytes) 2024-05-08T08:26:00.698 INFO:teuthology.orchestra.run.smithi031.stdout:4843 writesame 0x412c2e8 thru 0x413bff2 data_size 0x27(0xfd0b bytes) 2024-05-08T08:26:00.702 INFO:teuthology.orchestra.run.smithi031.stdout:4844 writesame 0x44e058b thru 0x44f04cd data_size 0x1(0xff43 bytes) 2024-05-08T08:26:00.719 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.719 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.719 INFO:teuthology.orchestra.run.smithi031.stdout:4848 writesame 0x2ef337d thru 0x2f013ee data_size 0x1(0xe072 bytes) 2024-05-08T08:26:00.736 INFO:teuthology.orchestra.run.smithi031.stdout:4849 writesame 0x4e10eb2 thru 0x4e1fefc data_size 0xf(0xf04b bytes) 2024-05-08T08:26:00.740 INFO:teuthology.orchestra.run.smithi031.stdout:4851 read 0xbc97e50 thru 0xbc9c50a (0x46bb bytes) 2024-05-08T08:26:00.740 INFO:teuthology.orchestra.run.smithi031.stdout:4852 read 0x929f429 thru 0x92a3ee2 (0x4aba bytes) 2024-05-08T08:26:00.741 INFO:teuthology.orchestra.run.smithi031.stdout:4853 read 0x59cbae0 thru 0x59d7399 (0xb8ba bytes) 2024-05-08T08:26:00.741 INFO:teuthology.orchestra.run.smithi031.stdout:4854 read 0x40cb00d thru 0x40d729d (0xc291 bytes) 2024-05-08T08:26:00.742 INFO:teuthology.orchestra.run.smithi031.stdout:4855 read 0x4f4b973 thru 0x4f4da4e (0x20dc bytes) 2024-05-08T08:26:00.742 INFO:teuthology.orchestra.run.smithi031.stdout:4856 read 0x92cda6e thru 0x92ce31d (0x8b0 bytes) 2024-05-08T08:26:00.742 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.742 INFO:teuthology.orchestra.run.smithi031.stdout:4858 write 0x2f52c54 thru 0x2f5d396 (0xa743 bytes) 2024-05-08T08:26:00.743 INFO:teuthology.orchestra.run.smithi031.stdout:4859 write 0x53f55a6 thru 0x53fae76 (0x58d1 bytes) 2024-05-08T08:26:00.745 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.753 INFO:teuthology.orchestra.run.smithi031.stdout:4861 write 0xd1bdb34 thru 0xd1c991a (0xbde7 bytes) 2024-05-08T08:26:00.776 INFO:teuthology.orchestra.run.smithi031.stdout:4862 trunc from 0xd1c991b to 0x13312a2 2024-05-08T08:26:00.781 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:00.789+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:00.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:00.812 INFO:teuthology.orchestra.run.smithi031.stdout:4863 writesame 0xa0fde83 thru 0xa10ba40 data_size 0xb(0xdbbe bytes) 2024-05-08T08:26:00.812 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.813 INFO:teuthology.orchestra.run.smithi031.stdout:4866 read 0x2a05d20 thru 0x2a0df56 (0x8237 bytes) 2024-05-08T08:26:00.824 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:00.824 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:00.824 INFO:teuthology.orchestra.run.smithi031.stdout:4869 writesame 0xd9f4aa4 thru 0xd9f9d21 data_size 0x1(0x527e bytes) 2024-05-08T08:26:00.837 INFO:teuthology.orchestra.run.smithi031.stdout:4870 trunc from 0xd9f9d22 to 0x2399c40 2024-05-08T08:26:00.839 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:00.849 INFO:teuthology.orchestra.run.smithi031.stdout:4871 punch from 0x1b1bf83 to 0x1b25b65, (0x9be2 bytes) 2024-05-08T08:26:00.853 INFO:teuthology.orchestra.run.smithi031.stdout:4873 trunc from 0x2399c40 to 0x5d9240d 2024-05-08T08:26:00.872 INFO:teuthology.orchestra.run.smithi031.stdout:4874 write 0x8838ec6 thru 0x883d051 (0x418c bytes) 2024-05-08T08:26:00.878 INFO:teuthology.orchestra.run.smithi031.stdout:4875 write 0x18b4f5 thru 0x190b71 (0x567d bytes) 2024-05-08T08:26:00.880 INFO:teuthology.orchestra.run.smithi031.stdout:4876 punch from 0x57d7afe to 0x57e3234, (0xb736 bytes) 2024-05-08T08:26:00.880 INFO:teuthology.orchestra.run.smithi031.stdout:4878 punch from 0x2fd4a46 to 0x2fd94a9, (0x4a63 bytes) 2024-05-08T08:26:00.880 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.882 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:00.890+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:00.884 INFO:teuthology.orchestra.run.smithi031.stdout:4883 trunc from 0x883d052 to 0x516d0ac 2024-05-08T08:26:00.920 INFO:teuthology.orchestra.run.smithi031.stdout:4885 writesame 0xe6b2173 thru 0xe6c0fd1 data_size 0x1(0xee5f bytes) 2024-05-08T08:26:00.921 INFO:teuthology.orchestra.run.smithi031.stdout:4886 punch from 0x6bbfa40 to 0x6bc937c, (0x993c bytes) 2024-05-08T08:26:00.921 INFO:teuthology.orchestra.run.smithi031.stdout:4887 read 0xa99953b thru 0xa99bc71 (0x2737 bytes) 2024-05-08T08:26:00.921 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:00.921 INFO:teuthology.orchestra.run.smithi031.stdout:4890 write 0x270f6b3 thru 0x271072e (0x107c bytes) 2024-05-08T08:26:00.928 INFO:teuthology.orchestra.run.smithi031.stdout:4892 clone 28 order 21 su 16384 sc 14 2024-05-08T08:26:00.933 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:00.933 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:00.953 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:00.953 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:00.970 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:00.970 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:00.973 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:00.973 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:00.973 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:00.983 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:00.984 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:00.990+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:00.989 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:00.989 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:01.007 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:01.008 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:01.026 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:01.026 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:01.043 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:01.044 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:01.061 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:01.062 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:01.101 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:01.101 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:01.101 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:01.102 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:01.108+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:01.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:01.193 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:01.201+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:01.226 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:26:01.269 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:01.269 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2370 3.4 e24/18 3:237be9c1:::rbd_data.119116cc7dd0.00000000000001eb:head v 24'46, mlcod=23'42)", 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 49.553868178999998, 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029578109999999999, 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721286+0000", 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721287+0000", 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7199999999999997e-07 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721294+0000", 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9800000000000001e-06 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721309+0000", 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5116999999999999e-05 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721317+0000", 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9030000000000004e-06 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721383+0000", 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6111000000000005e-05 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.721406+0000", 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2453e-05 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724192+0000", 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027864909999999999 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.724244+0000", 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1884e-05 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 49.549654871999998, 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 49.279961335000003, 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.250873568999999, 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.287 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.135721873000001, 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:01.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:01.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.293 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.012075451000001, 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:01.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:01.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.012004304000001, 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:01.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:26:01.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:26:01.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.011658574000002, 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:01.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:01.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:01.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.307 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.011617285, 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:26:01.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:26:01.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20624 3.4 3:21e3449a:::rbd_data.119174045df.0000000000000045:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.011318965999997, 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029912699999999999, 2024-05-08T08:26:01.312 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20624 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263836+0000", 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0200000000000001e-07 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263841+0000", 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4950000000000002e-06 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263854+0000", 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2937e-05 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263862+0000", 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6729999999999992e-06 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263949+0000", 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7406999999999998e-05 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263954+0000", 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4110000000000004e-06 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266741+0000", 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027867640000000002 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266827+0000", 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5780999999999997e-05 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.316 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21117 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 413696~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.797560036, 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003161652, 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21117 2024-05-08T08:26:01.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477596+0000", 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8050000000000001e-06 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477599+0000", 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.616e-06 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477604+0000", 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477608+0000", 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480366+0000", 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027578020000000002 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480420+0000", 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3735e-05 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480756+0000", 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033637199999999998 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.797537177999999, 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.321 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21738 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 450560~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.651024021000001, 2024-05-08T08:26:01.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031591710000000001, 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21738 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624133+0000", 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-05-08T08:26:01.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624140+0000", 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624145+0000", 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627084+0000", 2024-05-08T08:26:01.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029395129999999999 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627161+0000", 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7096999999999997e-05 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627290+0000", 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012866399999999999 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.505323348000001, 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.334 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.778121312, 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 30.559140946999999, 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:01.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:01.344 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.070915582000001, 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:01.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:01.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:01.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.150651427, 2024-05-08T08:26:01.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:01.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:01.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:01.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.002645293, 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:01.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 12.911784399, 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.365 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:01.367 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:01.367 INFO:teuthology.orchestra.run.smithi031.stdout:flattening image image_client.0-clone27 2024-05-08T08:26:01.367 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:01.374+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:01.467 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:01.475+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:01.470 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-05-08T08:26:01.535 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:01.535 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:01.535 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:01.547 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:01.588 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:01.595+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:01.670 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:01.677+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:01.677 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:01.677 INFO:teuthology.orchestra.run.smithi031.stdout:4896 read 0x5dc38d5 thru 0x5dd36a7 (0xfdd3 bytes) 2024-05-08T08:26:01.678 INFO:teuthology.orchestra.run.smithi031.stdout:4897 read 0x4ead695 thru 0x4eb4db0 (0x771c bytes) 2024-05-08T08:26:01.689 INFO:teuthology.orchestra.run.smithi031.stdout:4899 trunc from 0xe6c0fd2 to 0x5a63542 2024-05-08T08:26:01.704 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:01.705 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:01.715 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:01.716 INFO:teuthology.orchestra.run.smithi031.stdout:4900 write 0x50606f6 thru 0x506c9ca (0xc2d5 bytes) 2024-05-08T08:26:01.726 INFO:teuthology.orchestra.run.smithi031.stdout:4901 punch from 0x57c61d5 to 0x57d0fb2, (0xaddd bytes) 2024-05-08T08:26:01.734 INFO:teuthology.orchestra.run.smithi031.stdout:4902 write 0x7b381fd thru 0x7b47150 (0xef54 bytes) 2024-05-08T08:26:01.739 INFO:teuthology.orchestra.run.smithi031.stdout:4903 read 0x1be7fe6 thru 0x1bf39dc (0xb9f7 bytes) 2024-05-08T08:26:01.747 INFO:teuthology.orchestra.run.smithi031.stdout:4904 trunc from 0x7b47151 to 0x17d72c6 2024-05-08T08:26:01.771 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:01.778+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:01.872 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:01.879+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:01.876 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:01.876 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54375 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7232050640000001, 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060789163, 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.877 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54375 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160082+0000", 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.35e-06 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160085+0000", 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8430000000000001e-06 2024-05-08T08:26:01.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160088+0000", 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.332e-06 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220737+0000", 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060648805 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220808+0000", 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1247000000000004e-05 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220869+0000", 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1377999999999994e-05 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:01.881 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231940970000004, 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:01.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:26:01.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:26:01.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231840120000008, 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.889 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231735330000006, 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.890 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.892 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:26:01.893 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.723156607, 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.894 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:26:01.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7231356869999992, 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:26:01.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7230973719999998, 2024-05-08T08:26:01.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:26:01.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7230736259999997, 2024-05-08T08:26:01.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.910 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7230507829999997, 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:26:01.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7230263939999997, 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:26:01.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7230044739999997, 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:26:01.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:26:01.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7229832399999996, 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.923 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7229617959999999, 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.927 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:26:01.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219774920000006, 2024-05-08T08:26:01.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:26:01.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:26:01.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219562909999997, 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:01.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:26:01.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:26:01.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219448209999992, 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.939 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219350609999999, 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:01.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:26:01.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:26:01.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.721925272, 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:26:01.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219147479999997, 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.952 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:01.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:26:01.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.7219050679999999, 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:01.956 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:26:01.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:26:01.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:01.960 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:01.962 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:4906 write 0x1d31a39 thru 0x1d35b38 (0x4100 bytes) 2024-05-08T08:26:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:4907 write 0x45104c2 thru 0x4515da7 (0x58e6 bytes) 2024-05-08T08:26:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:4910 read 0x33a2c0b thru 0x33aa87b (0x7c71 bytes) 2024-05-08T08:26:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:4911 writesame 0x26c7eaf thru 0x26ccd45 data_size 0x1f(0x4e97 bytes) 2024-05-08T08:26:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:4914 write 0x6e22b6a thru 0x6e2cf2a (0xa3c1 bytes) 2024-05-08T08:26:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:4915 read 0x23ac54a thru 0x23b1c78 (0x572f bytes) 2024-05-08T08:26:01.963 INFO:teuthology.orchestra.run.smithi031.stdout:4917 trunc from 0x6e2cf2b to 0x26685ee 2024-05-08T08:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout:4918 writesame 0x2bc0afa thru 0x2bc9c11 data_size 0x4(0x9118 bytes) 2024-05-08T08:26:01.971 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:01.986 INFO:teuthology.orchestra.run.smithi031.stdout:4920 write 0xb10b117 thru 0xb10bfb8 (0xea2 bytes) 2024-05-08T08:26:01.997 INFO:teuthology.orchestra.run.smithi031.stdout:4921 trunc from 0xb10bfb9 to 0x4355a24 2024-05-08T08:26:02.006 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:02.014+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:02.007 INFO:teuthology.orchestra.run.smithi031.stdout:4923 punch from 0x1c4213b to 0x1c4f467, (0xd32c bytes) 2024-05-08T08:26:02.008 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.008 INFO:teuthology.orchestra.run.smithi031.stdout:4925 punch from 0x290ba82 to 0x290d37a, (0x18f8 bytes) 2024-05-08T08:26:02.021 INFO:teuthology.orchestra.run.smithi031.stdout:4926 write 0xc00b2c7 thru 0xc011d89 (0x6ac3 bytes) 2024-05-08T08:26:02.027 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.027 INFO:teuthology.orchestra.run.smithi031.stdout:4932 read 0x53129b thru 0x533e35 (0x2b9b bytes) 2024-05-08T08:26:02.028 INFO:teuthology.orchestra.run.smithi031.stdout:4933 write 0xa961959 thru 0xa96f97c (0xe024 bytes) 2024-05-08T08:26:02.040 INFO:teuthology.orchestra.run.smithi031.stdout:4934 writesame 0xc93bb47 thru 0xc949c8a data_size 0x1a(0xe144 bytes) 2024-05-08T08:26:02.040 INFO:teuthology.orchestra.run.smithi031.stdout:4935 read 0x73da86f thru 0x73e574a (0xaedc bytes) 2024-05-08T08:26:02.040 INFO:teuthology.orchestra.run.smithi031.stdout:4936 read 0x7717267 thru 0x772673c (0xf4d6 bytes) 2024-05-08T08:26:02.047 INFO:teuthology.orchestra.run.smithi031.stdout:4937 trunc from 0xc949c8b to 0x5e99485 2024-05-08T08:26:02.068 INFO:teuthology.orchestra.run.smithi031.stdout:4940 write 0x9445804 thru 0x945091e (0xb11b bytes) 2024-05-08T08:26:02.073 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:02.080+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:02.075 INFO:teuthology.orchestra.run.smithi031.stdout:4941 read 0x1c42aba thru 0x1c4d4a7 (0xa9ee bytes) 2024-05-08T08:26:02.075 INFO:teuthology.orchestra.run.smithi031.stdout:4942 write 0x94105a thru 0x944acb (0x3a72 bytes) 2024-05-08T08:26:02.076 INFO:teuthology.orchestra.run.smithi031.stdout:4943 write 0x8e1a116 thru 0x8e23d13 (0x9bfe bytes) 2024-05-08T08:26:02.083 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:02.083 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:02.083 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:02.083 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:02.092 INFO:teuthology.orchestra.run.smithi031.stdout:4946 writesame 0x9ad5b32 thru 0x9ae07f6 data_size 0x17(0xacc5 bytes) 2024-05-08T08:26:02.093 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:02.094 INFO:teuthology.orchestra.run.smithi031.stdout:4947 write 0x8a0b06a thru 0x8a1521b (0xa1b2 bytes) 2024-05-08T08:26:02.112 INFO:teuthology.orchestra.run.smithi031.stdout:4949 writesame 0xcba3c88 thru 0xcbad60b data_size 0x32(0x9984 bytes) 2024-05-08T08:26:02.122 INFO:teuthology.orchestra.run.smithi031.stdout:4950 trunc from 0xcbad60c to 0x361a7d3 2024-05-08T08:26:02.141 INFO:teuthology.orchestra.run.smithi031.stdout:4952 write 0xd8a4f4 thru 0xd93f8d (0x9a9a bytes) 2024-05-08T08:26:02.157 INFO:teuthology.orchestra.run.smithi031.stdout:4953 writesame 0xada01b5 thru 0xada8617 data_size 0x4f(0x8463 bytes) 2024-05-08T08:26:02.157 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.157 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.158 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.158 INFO:teuthology.orchestra.run.smithi031.stdout:4958 writesame 0x4c9103 thru 0x4d4e8b data_size 0xb(0xbd89 bytes) 2024-05-08T08:26:02.171 INFO:teuthology.orchestra.run.smithi031.stdout:4959 trunc from 0xada8618 to 0x298677d 2024-05-08T08:26:02.174 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:02.182+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:02.191 INFO:teuthology.orchestra.run.smithi031.stdout:4960 trunc from 0x298677d to 0xb4bc9e3 2024-05-08T08:26:02.197 INFO:teuthology.orchestra.run.smithi031.stdout:4962 punch from 0x350a89f to 0x3512a02, (0x8163 bytes) 2024-05-08T08:26:02.197 INFO:teuthology.orchestra.run.smithi031.stdout:4964 read 0x451ae37 thru 0x4522ffc (0x81c6 bytes) 2024-05-08T08:26:02.198 INFO:teuthology.orchestra.run.smithi031.stdout:4966 trunc from 0xb4bc9e3 to 0xbd1848a 2024-05-08T08:26:02.203 INFO:teuthology.orchestra.run.smithi031.stdout:4967 writesame 0x5d16404 thru 0x5d19cbd data_size 0x35(0x38ba bytes) 2024-05-08T08:26:02.210 INFO:teuthology.orchestra.run.smithi031.stdout:4968 punch from 0xaff26 to 0xb6490, (0x656a bytes) 2024-05-08T08:26:02.210 INFO:teuthology.orchestra.run.smithi031.stdout:4969 read 0x8019d4f thru 0x8021dcb (0x807d bytes) 2024-05-08T08:26:02.210 INFO:teuthology.orchestra.run.smithi031.stdout:4970 read 0x1e5de53 thru 0x1e6bbd3 (0xdd81 bytes) 2024-05-08T08:26:02.210 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:4973 punch from 0x8e5ac4f to 0x8e69b5d, (0xef0e bytes) 2024-05-08T08:26:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:4974 read 0xbb70d19 thru 0xbb746df (0x39c7 bytes) 2024-05-08T08:26:02.211 INFO:teuthology.orchestra.run.smithi031.stdout:4976 writesame 0x62ba90c thru 0x62c3406 data_size 0x1(0x8afb bytes) 2024-05-08T08:26:02.222 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:02.222 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:02.222 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T08:26:02.222 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-05-08T08:26:02.222 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0 2024-05-08T08:26:02.225 INFO:teuthology.orchestra.run.smithi031.stdout:4979 punch from 0x5432dac to 0x54364b1, (0x3705 bytes) 2024-05-08T08:26:02.226 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.226 INFO:teuthology.orchestra.run.smithi031.stdout:4981 read 0x35fac96 thru 0x35fb5fe (0x969 bytes) 2024-05-08T08:26:02.226 INFO:teuthology.orchestra.run.smithi031.stdout:4986 read 0x4f1305e thru 0x4f19718 (0x66bb bytes) 2024-05-08T08:26:02.226 INFO:teuthology.orchestra.run.smithi031.stdout:4987 read 0x70984ba thru 0x70a78c9 (0xf410 bytes) 2024-05-08T08:26:02.226 INFO:teuthology.orchestra.run.smithi031.stdout:4989 write 0x60f0fdb thru 0x60f4436 (0x345c bytes) 2024-05-08T08:26:02.232 INFO:teuthology.orchestra.run.smithi031.stdout:4990 punch from 0x49ae6cb to 0x49b6f9a, (0x88cf bytes) 2024-05-08T08:26:02.232 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.232 INFO:teuthology.orchestra.run.smithi031.stdout:4992 punch from 0x36c4610 to 0x36c9338, (0x4d28 bytes) 2024-05-08T08:26:02.233 INFO:teuthology.orchestra.run.smithi031.stdout:4993 write 0x48f0cfa thru 0x48f762c (0x6933 bytes) 2024-05-08T08:26:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:4995 read 0x46cf1dc thru 0x46dcfbf (0xdde4 bytes) 2024-05-08T08:26:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:4997 punch from 0x8ce2d7f to 0x8cf2a03, (0xfc84 bytes) 2024-05-08T08:26:02.237 INFO:teuthology.orchestra.run.smithi031.stdout:4999 writesame 0xbbd2ee8 thru 0xbbe0097 data_size 0x50(0xd1b0 bytes) 2024-05-08T08:26:02.247 INFO:teuthology.orchestra.run.smithi031.stdout:5000 read 0x9f27c9a thru 0x9f3310c (0xb473 bytes) 2024-05-08T08:26:02.247 INFO:teuthology.orchestra.run.smithi031.stdout:5001 punch from 0xa7c60b2 to 0xa7c9bd5, (0x3b23 bytes) 2024-05-08T08:26:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:5002 read 0x95d41c7 thru 0x95dfeed (0xbd27 bytes) 2024-05-08T08:26:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:5003 read 0x6f5da3 thru 0x6f8923 (0x2b81 bytes) 2024-05-08T08:26:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:5004 read 0x2a21232 thru 0x2a243e6 (0x31b5 bytes) 2024-05-08T08:26:02.248 INFO:teuthology.orchestra.run.smithi031.stdout:5005 writesame 0x27ffac5 thru 0x280c4c5 data_size 0x1(0xca01 bytes) 2024-05-08T08:26:02.251 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:02.251 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:02.251 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:02.251 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:02.251 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:02.265 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:02.268 INFO:teuthology.orchestra.run.smithi031.stdout:5006 read 0x3da404f thru 0x3daf032 (0xafe4 bytes) 2024-05-08T08:26:02.268 INFO:teuthology.orchestra.run.smithi031.stdout:5007 punch from 0x1112d89 to 0x1113000, (0x277 bytes) 2024-05-08T08:26:02.275 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:02.283+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:02.277 INFO:teuthology.orchestra.run.smithi031.stdout:5008 write 0xe855352 thru 0xe8566b8 (0x1367 bytes) 2024-05-08T08:26:02.280 INFO:teuthology.orchestra.run.smithi031.stdout:5009 write 0xc3c3539 thru 0xc3d0e5c (0xd924 bytes) 2024-05-08T08:26:02.287 INFO:teuthology.orchestra.run.smithi031.stdout:5010 punch from 0x7a13b2c to 0x7a20a03, (0xced7 bytes) 2024-05-08T08:26:02.287 INFO:teuthology.orchestra.run.smithi031.stdout:5011 read 0x318148c thru 0x3182ccf (0x1844 bytes) 2024-05-08T08:26:02.287 INFO:teuthology.orchestra.run.smithi031.stdout:5014 writesame 0x79bef18 thru 0x79c4d0a data_size 0x1(0x5df3 bytes) 2024-05-08T08:26:02.300 INFO:teuthology.orchestra.run.smithi031.stdout:5018 writesame 0xb41f7fb thru 0xb4219b4 data_size 0x3(0x21ba bytes) 2024-05-08T08:26:02.305 INFO:teuthology.orchestra.run.smithi031.stdout:5019 writesame 0xc5f0269 thru 0xc5fb7b0 data_size 0x4(0xb548 bytes) 2024-05-08T08:26:02.315 INFO:teuthology.orchestra.run.smithi031.stdout:5020 trunc from 0xe8566b9 to 0xc1eb3ce 2024-05-08T08:26:02.331 INFO:teuthology.orchestra.run.smithi031.stdout:5021 write 0xa03534a thru 0xa03d35d (0x8014 bytes) 2024-05-08T08:26:02.336 INFO:teuthology.orchestra.run.smithi031.stdout:5022 read 0xa1edd0f thru 0xa1f0806 (0x2af8 bytes) 2024-05-08T08:26:02.337 INFO:teuthology.orchestra.run.smithi031.stdout:5023 read 0xb5fc6a7 thru 0xb5ffa22 (0x337c bytes) 2024-05-08T08:26:02.337 INFO:teuthology.orchestra.run.smithi031.stdout:5024 write 0x2bcf2bb thru 0x2bd2e42 (0x3b88 bytes) 2024-05-08T08:26:02.351 INFO:teuthology.orchestra.run.smithi031.stdout:5025 trunc from 0xc1eb3ce to 0x16b4b28 2024-05-08T08:26:02.376 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:02.384+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:02.390 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.125706929000003, 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:02.391 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:26:02.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.396 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14789 3.9 3:90dc9cea:::rbd_data.119174045df.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 36.940608877999999, 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024649007000000001, 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14789 2024-05-08T08:26:02.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456476+0000", 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0279999999999999e-06 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456480+0000", 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1280000000000001e-06 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456491+0000", 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0436e-05 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456497+0000", 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.64e-06 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456545+0000", 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7432999999999998e-05 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456635+0000", 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9859999999999997e-05 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.460437+0000", 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038026510000000002 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481041+0000", 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020603652 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481053+0000", 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2301e-05 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:02.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481088+0000", 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4669999999999998e-05 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481124+0000", 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6208000000000003e-05 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23289 3.3 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:02.402 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.877467067000001, 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.028538101999999999, 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23289 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519618+0000", 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3170000000000001e-06 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522813+0000", 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031947130000000001 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522829+0000", 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6005e-05 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522837+0000", 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8990000000000001e-06 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522902+0000", 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4939999999999998e-05 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523060+0000", 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000158085 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523351+0000", 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00029110399999999998 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533281+0000", 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099297550000000002 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548076+0000", 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014795605999999999 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548119+0000", 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2993e-05 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548155+0000", 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5685000000000001e-05 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23675 3.9 3:93098200:::rbd_data.1191ea0f1b1f.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.392507682000002, 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.027687956, 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:02.408 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23675 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004578+0000", 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.198e-06 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004583+0000", 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5740000000000003e-06 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004593+0000", 2024-05-08T08:26:02.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0127e-05 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004599+0000", 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004650+0000", 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999997e-05 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004699+0000", 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9186e-05 2024-05-08T08:26:02.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.009797+0000", 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050987319999999999 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032181+0000", 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022383157000000001 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032193+0000", 2024-05-08T08:26:02.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2561999999999999e-05 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032228+0000", 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4332000000000003e-05 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032264+0000", 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6797e-05 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.413 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:32844 3.1a 3:5c5225a1:::rbd_object_map.11915c46bf0c:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 15.240012843000001, 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0015158669999999, 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 32844 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157073+0000", 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157079+0000", 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157093+0000", 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4107999999999999e-05 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157102+0000", 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0049999999999993e-06 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157167+0000", 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4935000000000001e-05 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157178+0000", 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158227+0000", 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.001049232 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158355+0000", 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012803000000000001 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158587+0000", 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023214899999999999 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49774 3.3 3:c428cbb1:::rbd_data.11914ad0077c.0000000000000004:1a [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.418 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.507361943999999, 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021265240000000001, 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49774 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:02.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0600000000000001e-07 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889724+0000", 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8980000000000001e-06 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889730+0000", 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889735+0000", 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7129999999999996e-06 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910823+0000", 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021087864000000001 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910898+0000", 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5555000000000004e-05 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910987+0000", 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9196000000000005e-05 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.422 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49825 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 187392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505875346, 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02117349, 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49825 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.423 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891211+0000", 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1019999999999999e-06 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891218+0000", 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8569999999999997e-06 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891223+0000", 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.741e-06 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912215+0000", 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020992055999999999 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912288+0000", 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3326999999999998e-05 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912382+0000", 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3980999999999993e-05 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.426 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49826 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 196608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505855189, 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021377759, 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49826 2024-05-08T08:26:02.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:02.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891231+0000", 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7540000000000001e-06 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891237+0000", 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7030000000000003e-06 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891242+0000", 2024-05-08T08:26:02.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0329999999999999e-06 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912410+0000", 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021168761000000001 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912484+0000", 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3837999999999993e-05 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912607+0000", 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012231000000000001 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49827 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 205824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505836264999999, 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021638398999999999, 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.431 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49827 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891250+0000", 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9130000000000001e-06 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891256+0000", 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6570000000000002e-06 2024-05-08T08:26:02.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891260+0000", 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5959999999999997e-06 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912651+0000", 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021390180000000002 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912727+0000", 2024-05-08T08:26:02.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6477000000000007e-05 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912886+0000", 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000159176 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49828 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 215040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505816628, 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021996597, 2024-05-08T08:26:02.435 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49828 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891268+0000", 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891270+0000", 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3640000000000002e-06 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891276+0000", 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6019999999999999e-06 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891281+0000", 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913075+0000", 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021794052000000001 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913159+0000", 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4345999999999995e-05 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913264+0000", 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000105071 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49829 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 224256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.439 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505797316000001, 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022644764000000001, 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49829 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891289+0000", 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9439999999999999e-06 2024-05-08T08:26:02.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891295+0000", 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.896e-06 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891300+0000", 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8350000000000003e-06 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913295+0000", 2024-05-08T08:26:02.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021995627 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913372+0000", 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6736000000000005e-05 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913931+0000", 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00055940299999999998 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49830 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505778393, 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022779863000000001, 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49830 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891308+0000", 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9319999999999999e-06 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891313+0000", 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.31e-06 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891318+0000", 2024-05-08T08:26:02.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5700000000000003e-06 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913980+0000", 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02266257 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914025+0000", 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4780999999999997e-05 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914085+0000", 2024-05-08T08:26:02.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0316999999999999e-05 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49831 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 242688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505760489, 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022998424999999999, 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49831 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891326+0000", 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.951e-06 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891332+0000", 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8200000000000002e-06 2024-05-08T08:26:02.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891336+0000", 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6789999999999996e-06 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914125+0000", 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022788283999999999 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914206+0000", 2024-05-08T08:26:02.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1599000000000003e-05 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914322+0000", 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00011571299999999999 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49832 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505742349, 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023170733999999998, 2024-05-08T08:26:02.452 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49832 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891344+0000", 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:26:02.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891350+0000", 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5679999999999999e-06 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891354+0000", 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5750000000000002e-06 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914351+0000", 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022996632999999999 2024-05-08T08:26:02.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914423+0000", 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2469e-05 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914512+0000", 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9166999999999995e-05 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49833 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.505724269, 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023398988999999999, 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49833 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:02.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891362+0000", 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8169999999999999e-06 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891367+0000", 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5110000000000003e-06 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891373+0000", 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5890000000000002e-06 2024-05-08T08:26:02.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914541+0000", 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023167982 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914614+0000", 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3317000000000003e-05 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.460 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914759+0000", 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00014440500000000001 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50084 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 216064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.498141097, 2024-05-08T08:26:02.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021113683000000001, 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50084 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898945+0000", 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9520000000000001e-06 2024-05-08T08:26:02.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898951+0000", 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898955+0000", 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8629999999999999e-06 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919894+0000", 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020938667000000001 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919965+0000", 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0927999999999996e-05 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920057+0000", 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1650999999999994e-05 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50085 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.465 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.497883012999999, 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021069139000000001, 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50085 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899202+0000", 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0999999999999997e-07 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899206+0000", 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.861e-06 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899214+0000", 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1289999999999996e-06 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899220+0000", 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5749999999999997e-06 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920083+0000", 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020862975999999998 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920177+0000", 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3540000000000002e-05 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920270+0000", 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3247999999999999e-05 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50086 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 234496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.497846855000001, 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021223872000000001, 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50086 2024-05-08T08:26:02.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899238+0000", 2024-05-08T08:26:02.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1099999999999995e-07 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899241+0000", 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8100000000000002e-06 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899247+0000", 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4139999999999999e-06 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899252+0000", 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:26:02.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920299+0000", 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021046914999999999 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920373+0000", 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3962999999999996e-05 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920461+0000", 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8419999999999997e-05 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.473 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50087 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.497824767999999, 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021404211999999999, 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50087 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899260+0000", 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0600000000000001e-07 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899262+0000", 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.058e-06 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899267+0000", 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6699999999999999e-06 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899272+0000", 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6920000000000001e-06 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920490+0000", 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021217881000000001 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920561+0000", 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1377000000000005e-05 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920663+0000", 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102128 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50088 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.497804524999999, 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021585928000000001, 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.478 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50088 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899280+0000", 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899282+0000", 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1330000000000002e-06 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899288+0000", 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6890000000000001e-06 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899293+0000", 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8160000000000001e-06 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920704+0000", 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021411660999999999 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920778+0000", 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3788999999999994e-05 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920865+0000", 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7437999999999999e-05 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.482 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:02.484 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:5026 write 0x70182a3 thru 0x7021d9a (0x9af8 bytes) 2024-05-08T08:26:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:5027 trunc from 0x7021d9b to 0x88992e 2024-05-08T08:26:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:5029 read 0x242395 thru 0x251f87 (0xfbf3 bytes) 2024-05-08T08:26:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:5030 read 0x35f563 thru 0x3668c6 (0x7364 bytes) 2024-05-08T08:26:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:5032 write 0x4276275 thru 0x427af68 (0x4cf4 bytes) 2024-05-08T08:26:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:5033 read 0x15427cd thru 0x15435f8 (0xe2c bytes) 2024-05-08T08:26:02.486 INFO:teuthology.orchestra.run.smithi031.stdout:5034 write 0x12bae02 thru 0x12c1023 (0x6222 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5035 write 0x70a086c thru 0x70a3862 (0x2ff7 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5036 trunc from 0x70a3863 to 0xd885b52 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5039 writesame 0x460f57a thru 0x461ba52 data_size 0x17(0xc4d9 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5040 read 0xd6d8ceb thru 0xd6d992a (0xc40 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5042 read 0xb7df3b0 thru 0xb7ec573 (0xd1c4 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5044 write 0x1ad9226 thru 0x1ae293f (0x971a bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5045 punch from 0x908bb61 to 0x9096c30, (0xb0cf bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5047 read 0x866eec thru 0x868275 (0x138a bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5048 write 0x65e0e78 thru 0x65e13db (0x564 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5050 read 0x78734f6 thru 0x7873a1a (0x525 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5051 read 0xd4bec47 thru 0xd4c77e9 (0x8ba3 bytes) 2024-05-08T08:26:02.487 INFO:teuthology.orchestra.run.smithi031.stdout:5052 write 0xd65e704 thru 0xd65f79f (0x109c bytes) 2024-05-08T08:26:02.488 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:02.491+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:02.488 INFO:teuthology.orchestra.run.smithi031.stdout:5054 trunc from 0xd885b52 to 0x5a3a6e8 2024-05-08T08:26:02.510 INFO:teuthology.orchestra.run.smithi031.stdout:5055 writesame 0xa120932 thru 0xa12b269 data_size 0x1e(0xa938 bytes) 2024-05-08T08:26:02.510 INFO:teuthology.orchestra.run.smithi031.stdout:5056 write 0x2949043 thru 0x2951795 (0x8753 bytes) 2024-05-08T08:26:02.518 INFO:teuthology.orchestra.run.smithi031.stdout:5059 trunc from 0xa12b26a to 0x9a0a5fa 2024-05-08T08:26:02.523 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.524 INFO:teuthology.orchestra.run.smithi031.stdout:5061 writesame 0x5eccb24 thru 0x5ecfdbd data_size 0x3(0x329a bytes) 2024-05-08T08:26:02.527 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.527 INFO:teuthology.orchestra.run.smithi031.stdout:5064 write 0xd04149 thru 0xd122e0 (0xe198 bytes) 2024-05-08T08:26:02.529 INFO:teuthology.orchestra.run.smithi031.stdout:5065 write 0x85c5d48 thru 0x85d4fd3 (0xf28c bytes) 2024-05-08T08:26:02.537 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.537 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.537 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:02.537 INFO:teuthology.orchestra.run.smithi031.stdout:5069 punch from 0x45a6db2 to 0x45b0177, (0x93c5 bytes) 2024-05-08T08:26:02.537 INFO:teuthology.orchestra.run.smithi031.stdout:5072 read 0x7a8a9 thru 0x8158c (0x6ce4 bytes) 2024-05-08T08:26:02.537 INFO:teuthology.orchestra.run.smithi031.stdout:5073 punch from 0xfe3490 to 0xfeb094, (0x7c04 bytes) 2024-05-08T08:26:02.546 INFO:teuthology.orchestra.run.smithi031.stdout:5077 write 0xca82e85 thru 0xca87394 (0x4510 bytes) 2024-05-08T08:26:02.550 INFO:teuthology.orchestra.run.smithi031.stdout:5079 write 0x15d5165 thru 0x15db4ad (0x6349 bytes) 2024-05-08T08:26:02.552 INFO:teuthology.orchestra.run.smithi031.stdout:5080 writesame 0x54caee2 thru 0x54d92e7 data_size 0x5e(0xe406 bytes) 2024-05-08T08:26:02.569 INFO:teuthology.orchestra.run.smithi031.stdout:5081 write 0xd9717e9 thru 0xd979182 (0x799a bytes) 2024-05-08T08:26:02.579 INFO:teuthology.orchestra.run.smithi031.stdout:5082 punch from 0x4e9eb70 to 0x4ea77c1, (0x8c51 bytes) 2024-05-08T08:26:02.579 INFO:teuthology.orchestra.run.smithi031.stdout:5083 punch from 0x26134c2 to 0x261d805, (0xa343 bytes) 2024-05-08T08:26:02.579 INFO:teuthology.orchestra.run.smithi031.stdout:5086 punch from 0x2b736b5 to 0x2b7b159, (0x7aa4 bytes) 2024-05-08T08:26:02.580 INFO:teuthology.orchestra.run.smithi031.stdout:5089 read 0x5e1d89b thru 0x5e26836 (0x8f9c bytes) 2024-05-08T08:26:02.580 INFO:teuthology.orchestra.run.smithi031.stdout:5091 read 0xc801fb6 thru 0xc808cfb (0x6d46 bytes) 2024-05-08T08:26:02.580 INFO:teuthology.orchestra.run.smithi031.stdout:5093 clone 29 order 21 su 131072 sc 8 2024-05-08T08:26:02.585 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:02.592+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:02.637 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:02.637 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:02.638 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:02.638 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:02.646 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:02.690 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:02.698+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:02.753 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:02.753 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:02.753 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:02.753 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:02.753 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:02.762 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:02.804 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:02.811+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:02.888 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:02.896+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 33.257829352999998, 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:02.912 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:02.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.004348331999999, 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.918 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.004296384, 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.926 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.004261529000001, 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:02.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.004231265, 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:02.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.002921899, 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.935 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.002901137, 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.939 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:02.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:02.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.002829667, 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:02.943 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.947 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.002802299000001, 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:02.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:02.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:02.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.951 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.000847909000001, 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.952 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:02.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.000828342, 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.956 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:02.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:02.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.000808572, 2024-05-08T08:26:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:02.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:02.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.964 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 11.000789574000001, 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:02.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:02.968 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.998889938, 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.969 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:02.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.998869971, 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.973 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:02.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:02.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.998848398, 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:02.977 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:02.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:02.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.998724407999999, 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.998695613000001, 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.986 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:02.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:02.991 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.996778076, 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:02.995 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 10.996756621999999, 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:02.999 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:03.001 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:03.001 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:03.008+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:03.074 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:03.102 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:03.109+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:03.149 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:03.149 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:03.158 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:03.203 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:03.210+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:03.210 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:03.210 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:03.228 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:03.228 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:03.238 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 0 (802) 2024-05-08T08:26:03.244 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:26:03.249 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:03.250 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:03.268 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:03.269 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:03.286 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:03.286 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:03.304 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:03.311+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:03.304 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:03.305 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:03.309 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:03.309 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:03.309 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:03.309 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:03.309 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:03.318 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:03.322 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:03.322 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:03.340 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:03.340 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:03.405 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:03.412+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:03.465 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:03.465 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:03.473 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:03.505 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:03.513+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:03.578 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:03.579 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:03.587 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:03.607 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:03.614+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:03.693 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:03.693 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:03.702 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:03.745 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:03.753+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:03.808 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:03.808 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:03.809 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:03.816+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:03.817 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:03.909 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:03.916+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:03.930 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:03.930 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:03.939 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:04.022 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:04.030+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:04.088 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:04.088 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:04.097 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:04.110 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:04.118+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:04.211 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:04.218+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:04.236 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:26:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:04.252 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:04.261 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:04.300 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone28 from 0xd979183 (overlap 0x88992e) to 0x0 2024-05-08T08:26:04.343 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:04.351+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:04.373 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-05-08T08:26:04.413 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:04.420+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:04.424 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:04.424 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:04.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:04.520 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:04.528+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:04.590 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:04.590 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:04.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:04.623 INFO:teuthology.orchestra.run.smithi031.stdout:5094 write 0x8720aa4 thru 0x872ed3b (0xe298 bytes) 2024-05-08T08:26:04.623 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:04.630+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:04.634 INFO:teuthology.orchestra.run.smithi031.stdout:5095 punch from 0x3932421 to 0x3939207, (0x6de6 bytes) 2024-05-08T08:26:04.644 INFO:teuthology.orchestra.run.smithi031.stdout:5096 trunc from 0xd979183 to 0x4c43b55 2024-05-08T08:26:04.698 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:04.698 INFO:teuthology.orchestra.run.smithi031.stdout:5099 punch from 0x41a1c95 to 0x41a2ffb, (0x1366 bytes) 2024-05-08T08:26:04.709 INFO:teuthology.orchestra.run.smithi031.stdout:5100 trunc from 0x4c43b55 to 0xe5ce2b5 2024-05-08T08:26:04.715 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:04.722+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:04.715 INFO:teuthology.orchestra.run.smithi031.stdout:5101 trunc from 0xe5ce2b5 to 0xe955d3b 2024-05-08T08:26:04.721 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:04.721 INFO:teuthology.orchestra.run.smithi031.stdout:5105 read 0x5641669 thru 0x564afcf (0x9967 bytes) 2024-05-08T08:26:04.721 INFO:teuthology.orchestra.run.smithi031.stdout:5107 write 0xa42ba2e thru 0xa439be2 (0xe1b5 bytes) 2024-05-08T08:26:04.724 INFO:teuthology.orchestra.run.smithi031.stdout:5108 punch from 0x6b8d6f7 to 0x6b8ef1e, (0x1827 bytes) 2024-05-08T08:26:04.737 INFO:teuthology.orchestra.run.smithi031.stdout:5109 trunc from 0xe955d3b to 0x3cb19b3 2024-05-08T08:26:04.767 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:04.767 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:04.777 INFO:teuthology.orchestra.run.smithi031.stdout:5112 trunc from 0x3cb19b3 to 0x68f9006 2024-05-08T08:26:04.778 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:04.782 INFO:teuthology.orchestra.run.smithi031.stdout:5115 writesame 0x27d0b20 thru 0x27d3e28 data_size 0x41(0x3309 bytes) 2024-05-08T08:26:04.788 INFO:teuthology.orchestra.run.smithi031.stdout:5117 punch from 0x3daa554 to 0x3db6074, (0xbb20 bytes) 2024-05-08T08:26:04.799 INFO:teuthology.orchestra.run.smithi031.stdout:5118 write 0xba57e2e thru 0xba61a4e (0x9c21 bytes) 2024-05-08T08:26:04.803 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:04.811 INFO:teuthology.orchestra.run.smithi031.stdout:5120 write 0xd34db8c thru 0xd34ee89 (0x12fe bytes) 2024-05-08T08:26:04.816 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:04.823+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:04.820 INFO:teuthology.orchestra.run.smithi031.stdout:5121 writesame 0xe5a7b63 thru 0xe5b0da4 data_size 0x61(0x9242 bytes) 2024-05-08T08:26:04.820 INFO:teuthology.orchestra.run.smithi031.stdout:5122 read 0xc384e88 thru 0xc388aa8 (0x3c21 bytes) 2024-05-08T08:26:04.820 INFO:teuthology.orchestra.run.smithi031.stdout:5123 writesame 0xb783c9a thru 0xb787b70 data_size 0x1(0x3ed7 bytes) 2024-05-08T08:26:04.840 INFO:teuthology.orchestra.run.smithi031.stdout:5124 trunc from 0xe5b0da5 to 0x4a8cc13 2024-05-08T08:26:04.850 INFO:teuthology.orchestra.run.smithi031.stdout:5125 read 0x4618ef9 thru 0x4624452 (0xb55a bytes) 2024-05-08T08:26:04.850 INFO:teuthology.orchestra.run.smithi031.stdout:5126 read 0x4013fc7 thru 0x401e22b (0xa265 bytes) 2024-05-08T08:26:04.850 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:04.850 INFO:teuthology.orchestra.run.smithi031.stdout:5128 read 0x23132fe thru 0x231fdf2 (0xcaf5 bytes) 2024-05-08T08:26:04.851 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:04.851 INFO:teuthology.orchestra.run.smithi031.stdout:5130 read 0x15d9d98 thru 0x15db759 (0x19c2 bytes) 2024-05-08T08:26:04.851 INFO:teuthology.orchestra.run.smithi031.stdout:5131 read 0x2bbd284 thru 0x2bc7d9a (0xab17 bytes) 2024-05-08T08:26:04.866 INFO:teuthology.orchestra.run.smithi031.stdout:5132 write 0xc466ba5 thru 0xc469b81 (0x2fdd bytes) 2024-05-08T08:26:04.878 INFO:teuthology.orchestra.run.smithi031.stdout:5133 writesame 0xc81c700 thru 0xc82c50a data_size 0x1(0xfe0b bytes) 2024-05-08T08:26:04.878 INFO:teuthology.orchestra.run.smithi031.stdout:5135 read 0xb6643dd thru 0xb665ba1 (0x17c5 bytes) 2024-05-08T08:26:04.878 INFO:teuthology.orchestra.run.smithi031.stdout:5136 write 0x84cfe6e thru 0x84dd9bd (0xdb50 bytes) 2024-05-08T08:26:04.881 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:04.881 INFO:teuthology.orchestra.run.smithi031.stdout:5139 writesame 0x6362385 thru 0x636432a data_size 0x1(0x1fa6 bytes) 2024-05-08T08:26:04.888 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:04.888 INFO:teuthology.orchestra.run.smithi031.stdout:5141 punch from 0x684245 to 0x68792d, (0x36e8 bytes) 2024-05-08T08:26:04.888 INFO:teuthology.orchestra.run.smithi031.stdout:5142 write 0x45c7e98 thru 0x45ca871 (0x29da bytes) 2024-05-08T08:26:04.891 INFO:teuthology.orchestra.run.smithi031.stdout:5144 read 0x7ddb0d0 thru 0x7dde214 (0x3145 bytes) 2024-05-08T08:26:04.891 INFO:teuthology.orchestra.run.smithi031.stdout:5145 write 0xc51f35c thru 0xc52a772 (0xb417 bytes) 2024-05-08T08:26:04.895 INFO:teuthology.orchestra.run.smithi031.stdout:5149 read 0x1df168c thru 0x1dfca90 (0xb405 bytes) 2024-05-08T08:26:04.895 INFO:teuthology.orchestra.run.smithi031.stdout:5150 read 0x6605d5d thru 0x6612ff5 (0xd299 bytes) 2024-05-08T08:26:04.895 INFO:teuthology.orchestra.run.smithi031.stdout:5151 punch from 0x8ba08b7 to 0x8ba1061, (0x7aa bytes) 2024-05-08T08:26:04.895 INFO:teuthology.orchestra.run.smithi031.stdout:5152 punch from 0x8902fb5 to 0x89044a2, (0x14ed bytes) 2024-05-08T08:26:04.898 INFO:teuthology.orchestra.run.smithi031.stdout:5153 trunc from 0xc82c50b to 0xed479f5 2024-05-08T08:26:04.903 INFO:teuthology.orchestra.run.smithi031.stdout:5154 read 0x6360083 thru 0x6362822 (0x27a0 bytes) 2024-05-08T08:26:04.904 INFO:teuthology.orchestra.run.smithi031.stdout:5157 flatten 2024-05-08T08:26:04.910 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:04.910 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:04.917 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:04.924+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:04.926 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:05.011 INFO:teuthology.orchestra.run.smithi031.stdout:5158 punch from 0x4b26159 to 0x4b27ee0, (0x1d87 bytes) 2024-05-08T08:26:05.011 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.011 INFO:teuthology.orchestra.run.smithi031.stdout:5160 punch from 0x41c95c4 to 0x41d59b9, (0xc3f5 bytes) 2024-05-08T08:26:05.011 INFO:teuthology.orchestra.run.smithi031.stdout:5161 read 0xddfa481 thru 0xde03b22 (0x96a2 bytes) 2024-05-08T08:26:05.011 INFO:teuthology.orchestra.run.smithi031.stdout:5162 write 0x212f5c9 thru 0x2137846 (0x827e bytes) 2024-05-08T08:26:05.015 INFO:teuthology.orchestra.run.smithi031.stdout:5163 write 0xdc8f074 thru 0xdc936c7 (0x4654 bytes) 2024-05-08T08:26:05.017 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:05.025+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:05.020 INFO:teuthology.orchestra.run.smithi031.stdout:5164 read 0xcba8626 thru 0xcbad37c (0x4d57 bytes) 2024-05-08T08:26:05.020 INFO:teuthology.orchestra.run.smithi031.stdout:5165 punch from 0x32ba476 to 0x32c7fc3, (0xdb4d bytes) 2024-05-08T08:26:05.033 INFO:teuthology.orchestra.run.smithi031.stdout:5166 trunc from 0xed479f5 to 0x28e5018 2024-05-08T08:26:05.070 INFO:teuthology.orchestra.run.smithi031.stdout:5167 write 0xce0ad53 thru 0xce1a4b4 (0xf762 bytes) 2024-05-08T08:26:05.074 INFO:teuthology.orchestra.run.smithi031.stdout:5169 write 0x7617ca6 thru 0x762500f (0xd36a bytes) 2024-05-08T08:26:05.077 INFO:teuthology.orchestra.run.smithi031.stdout:5170 write 0xc4a9ca2 thru 0xc4ae746 (0x4aa5 bytes) 2024-05-08T08:26:05.080 INFO:teuthology.orchestra.run.smithi031.stdout:5172 writesame 0xbbd7545 thru 0xbbe0c75 data_size 0x1(0x9731 bytes) 2024-05-08T08:26:05.094 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:05.094 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:05.099 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.099 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.099 INFO:teuthology.orchestra.run.smithi031.stdout:5178 read 0xc345698 thru 0xc3493a0 (0x3d09 bytes) 2024-05-08T08:26:05.100 INFO:teuthology.orchestra.run.smithi031.stdout:5180 read 0x142715c thru 0x142ff71 (0x8e16 bytes) 2024-05-08T08:26:05.102 INFO:teuthology.orchestra.run.smithi031.stdout:5181 trunc from 0xce1a4b5 to 0xa34d43d 2024-05-08T08:26:05.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:05.112 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.112 INFO:teuthology.orchestra.run.smithi031.stdout:5185 writesame 0x5903bc0 thru 0x590b438 data_size 0x1(0x7879 bytes) 2024-05-08T08:26:05.118 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:05.126+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:05.137 INFO:teuthology.orchestra.run.smithi031.stdout:5187 trunc from 0xa34d43d to 0x36081 2024-05-08T08:26:05.156 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.156 INFO:teuthology.orchestra.run.smithi031.stdout:5189 read 0x1200c thru 0x14b86 (0x2b7b bytes) 2024-05-08T08:26:05.162 INFO:teuthology.orchestra.run.smithi031.stdout:5191 trunc from 0x36081 to 0x4eefa92 2024-05-08T08:26:05.168 INFO:teuthology.orchestra.run.smithi031.stdout:5193 punch from 0x284b421 to 0x2858071, (0xcc50 bytes) 2024-05-08T08:26:05.182 INFO:teuthology.orchestra.run.smithi031.stdout:5194 write 0xc1cbf0e thru 0xc1d7044 (0xb137 bytes) 2024-05-08T08:26:05.185 INFO:teuthology.orchestra.run.smithi031.stdout:5195 write 0x4266fc8 thru 0x42750df (0xe118 bytes) 2024-05-08T08:26:05.197 INFO:teuthology.orchestra.run.smithi031.stdout:5197 writesame 0xe5f16de thru 0xe5fc2be data_size 0x3(0xabe1 bytes) 2024-05-08T08:26:05.197 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.197 INFO:teuthology.orchestra.run.smithi031.stdout:5200 read 0x1af32c8 thru 0x1af9187 (0x5ec0 bytes) 2024-05-08T08:26:05.197 INFO:teuthology.orchestra.run.smithi031.stdout:5201 read 0xbc9341b thru 0xbc9d8b6 (0xa49c bytes) 2024-05-08T08:26:05.197 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.197 INFO:teuthology.orchestra.run.smithi031.stdout:5203 write 0x9e34240 thru 0x9e3f522 (0xb2e3 bytes) 2024-05-08T08:26:05.202 INFO:teuthology.orchestra.run.smithi031.stdout:5204 trunc from 0xe5fc2bf to 0xc5302ed 2024-05-08T08:26:05.209 INFO:teuthology.orchestra.run.smithi031.stdout:5205 writesame 0x8702429 thru 0x870f7c7 data_size 0x37(0xd39f bytes) 2024-05-08T08:26:05.219 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:05.226+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:05.222 INFO:teuthology.orchestra.run.smithi031.stdout:5206 trunc from 0xc5302ed to 0x317e64f 2024-05-08T08:26:05.226 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:05.226 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:05.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:05.246 INFO:teuthology.orchestra.run.smithi031.stdout:5207 write 0xa28ab3b thru 0xa299a68 (0xef2e bytes) 2024-05-08T08:26:05.251 INFO:teuthology.orchestra.run.smithi031.stdout:5208 read 0x3e584bf thru 0x3e5c046 (0x3b88 bytes) 2024-05-08T08:26:05.258 INFO:teuthology.orchestra.run.smithi031.stdout:5209 trunc from 0xa299a69 to 0x2cc38d6 2024-05-08T08:26:05.268 INFO:teuthology.orchestra.run.smithi031.stdout:5210 read 0x1a83fc7 thru 0x1a86b8d (0x2bc7 bytes) 2024-05-08T08:26:05.275 INFO:teuthology.orchestra.run.smithi031.stdout:5211 write 0x32f3db2 thru 0x32f9d08 (0x5f57 bytes) 2024-05-08T08:26:05.278 INFO:teuthology.orchestra.run.smithi031.stdout:5213 read 0x2570187 thru 0x257280d (0x2687 bytes) 2024-05-08T08:26:05.288 INFO:teuthology.orchestra.run.smithi031.stdout:5214 write 0x7a9e2a3 thru 0x7aa4032 (0x5d90 bytes) 2024-05-08T08:26:05.292 INFO:teuthology.orchestra.run.smithi031.stdout:5215 read 0x4c18133 thru 0x4c1f887 (0x7755 bytes) 2024-05-08T08:26:05.297 INFO:teuthology.orchestra.run.smithi031.stdout:5217 trunc from 0x7aa4033 to 0xc11d126 2024-05-08T08:26:05.302 INFO:teuthology.orchestra.run.smithi031.stdout:5218 read 0x80adb53 thru 0x80b3d1c (0x61ca bytes) 2024-05-08T08:26:05.303 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.303 INFO:teuthology.orchestra.run.smithi031.stdout:5220 write 0xbbbdc99 thru 0xbbc12e2 (0x364a bytes) 2024-05-08T08:26:05.306 INFO:teuthology.orchestra.run.smithi031.stdout:5221 punch from 0x3709100 to 0x3712088, (0x8f88 bytes) 2024-05-08T08:26:05.306 INFO:teuthology.orchestra.run.smithi031.stdout:5222 read 0xb6853a8 thru 0xb68f149 (0x9da2 bytes) 2024-05-08T08:26:05.306 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.307 INFO:teuthology.orchestra.run.smithi031.stdout:5226 read 0x8b2cbf2 thru 0x8b2fb39 (0x2f48 bytes) 2024-05-08T08:26:05.307 INFO:teuthology.orchestra.run.smithi031.stdout:5227 read 0xba8985c thru 0xba98672 (0xee17 bytes) 2024-05-08T08:26:05.307 INFO:teuthology.orchestra.run.smithi031.stdout:5228 punch from 0x8d426fc to 0x8d4dca0, (0xb5a4 bytes) 2024-05-08T08:26:05.315 INFO:teuthology.orchestra.run.smithi031.stdout:5232 trunc from 0xc11d126 to 0x477f93e 2024-05-08T08:26:05.319 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:05.327+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:05.324 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.331 INFO:teuthology.orchestra.run.smithi031.stdout:5234 write 0x5002149 thru 0x500f692 (0xd54a bytes) 2024-05-08T08:26:05.334 INFO:teuthology.orchestra.run.smithi031.stdout:5235 read 0x24ac203 thru 0x24bae66 (0xec64 bytes) 2024-05-08T08:26:05.348 INFO:teuthology.orchestra.run.smithi031.stdout:5236 write 0xbfd17e6 thru 0xbfd4c23 (0x343e bytes) 2024-05-08T08:26:05.350 INFO:teuthology.orchestra.run.smithi031.stdout:5238 write 0x24ae85a thru 0x24b11db (0x2982 bytes) 2024-05-08T08:26:05.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:05.354 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.354 INFO:teuthology.orchestra.run.smithi031.stdout:5240 writesame 0x6550aa2 thru 0x65603d0 data_size 0xd(0xf92f bytes) 2024-05-08T08:26:05.361 INFO:teuthology.orchestra.run.smithi031.stdout:5242 punch from 0x431bf25 to 0x4325ce3, (0x9dbe bytes) 2024-05-08T08:26:05.367 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:05.367 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:05.369 INFO:teuthology.orchestra.run.smithi031.stdout:5243 write 0xdd36f11 thru 0xdd3a280 (0x3370 bytes) 2024-05-08T08:26:05.372 INFO:teuthology.orchestra.run.smithi031.stdout:5244 write 0x8f57c3b thru 0x8f60fa0 (0x9366 bytes) 2024-05-08T08:26:05.377 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:05.378 INFO:teuthology.orchestra.run.smithi031.stdout:5245 writesame 0x968e3a0 thru 0x9692591 data_size 0x17(0x41f2 bytes) 2024-05-08T08:26:05.381 INFO:teuthology.orchestra.run.smithi031.stdout:5246 write 0x2ce54cb thru 0x2cf2539 (0xd06f bytes) 2024-05-08T08:26:05.399 INFO:teuthology.orchestra.run.smithi031.stdout:5250 trunc from 0xdd3a281 to 0x13ac9ba 2024-05-08T08:26:05.421 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:05.429+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:05.423 INFO:teuthology.orchestra.run.smithi031.stdout:5251 writesame 0x81efb3b thru 0x81f7fdd data_size 0x1(0x84a3 bytes) 2024-05-08T08:26:05.423 INFO:teuthology.orchestra.run.smithi031.stdout:5253 write 0x50ecdc4 thru 0x50f0863 (0x3aa0 bytes) 2024-05-08T08:26:05.432 INFO:teuthology.orchestra.run.smithi031.stdout:5254 trunc from 0x81f7fde to 0x2d99ea6 2024-05-08T08:26:05.442 INFO:teuthology.orchestra.run.smithi031.stdout:5255 punch from 0x102e568 to 0x102ed6c, (0x804 bytes) 2024-05-08T08:26:05.445 INFO:teuthology.orchestra.run.smithi031.stdout:5257 trunc from 0x2d99ea6 to 0xce2208 2024-05-08T08:26:05.470 INFO:teuthology.orchestra.run.smithi031.stdout:5258 write 0xc1d0bdf thru 0xc1d9fdc (0x93fe bytes) 2024-05-08T08:26:05.473 INFO:teuthology.orchestra.run.smithi031.stdout:5260 write 0xa468491 thru 0xa473868 (0xb3d8 bytes) 2024-05-08T08:26:05.476 INFO:teuthology.orchestra.run.smithi031.stdout:5262 write 0x58237bc thru 0x5827871 (0x40b6 bytes) 2024-05-08T08:26:05.479 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.480 INFO:teuthology.orchestra.run.smithi031.stdout:5265 read 0x47982e6 thru 0x47a3876 (0xb591 bytes) 2024-05-08T08:26:05.480 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.480 INFO:teuthology.orchestra.run.smithi031.stdout:5267 read 0x6107bed thru 0x6109d09 (0x211d bytes) 2024-05-08T08:26:05.480 INFO:teuthology.orchestra.run.smithi031.stdout:5268 read 0x74c1f9e thru 0x74d1203 (0xf266 bytes) 2024-05-08T08:26:05.480 INFO:teuthology.orchestra.run.smithi031.stdout:5269 punch from 0xa521796 to 0xa5266c9, (0x4f33 bytes) 2024-05-08T08:26:05.482 INFO:teuthology.orchestra.run.smithi031.stdout:5270 trunc from 0xc1d9fdd to 0xa0620e7 2024-05-08T08:26:05.493 INFO:teuthology.orchestra.run.smithi031.stdout:5271 write 0x2f0c70b thru 0x2f0f41f (0x2d15 bytes) 2024-05-08T08:26:05.498 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:05.499 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:05.499 INFO:teuthology.orchestra.run.smithi031.stdout:5272 trunc from 0xa0620e7 to 0x7cb3ce6 2024-05-08T08:26:05.504 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:05.504 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:05.508 INFO:teuthology.orchestra.run.smithi031.stdout:5273 writesame 0x52a979c thru 0x52aabd2 data_size 0x19(0x1437 bytes) 2024-05-08T08:26:05.512 INFO:teuthology.orchestra.run.smithi031.stdout:5274 write 0x1e5e81b thru 0x1e6db08 (0xf2ee bytes) 2024-05-08T08:26:05.514 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:05.516 INFO:teuthology.orchestra.run.smithi031.stdout:5275 punch from 0x7c3a84b to 0x7c3c6a4, (0x1e59 bytes) 2024-05-08T08:26:05.516 INFO:teuthology.orchestra.run.smithi031.stdout:5278 write 0x1a6436 thru 0x1b4e7d (0xea48 bytes) 2024-05-08T08:26:05.518 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:05.519 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:05.519 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.519 INFO:teuthology.orchestra.run.smithi031.stdout:5281 read 0x513212d thru 0x513dca7 (0xbb7b bytes) 2024-05-08T08:26:05.522 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:05.530+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:05.525 INFO:teuthology.orchestra.run.smithi031.stdout:5282 trunc from 0x7cb3ce6 to 0xced8c83 2024-05-08T08:26:05.531 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.531 INFO:teuthology.orchestra.run.smithi031.stdout:5285 writesame 0x2140e7c thru 0x214e7d6 data_size 0x1(0xd95b bytes) 2024-05-08T08:26:05.538 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:05.538 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:05.555 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:05.556 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:05.556 INFO:teuthology.orchestra.run.smithi031.stdout:5286 write 0x6c265a7 thru 0x6c270d1 (0xb2b bytes) 2024-05-08T08:26:05.560 INFO:teuthology.orchestra.run.smithi031.stdout:5287 punch from 0xa47a9d1 to 0xa48a0c4, (0xf6f3 bytes) 2024-05-08T08:26:05.560 INFO:teuthology.orchestra.run.smithi031.stdout:5288 read 0xc2fc27b thru 0xc2fdf29 (0x1caf bytes) 2024-05-08T08:26:05.560 INFO:teuthology.orchestra.run.smithi031.stdout:5289 write 0x682c70b thru 0x682da43 (0x1339 bytes) 2024-05-08T08:26:05.561 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.561 INFO:teuthology.orchestra.run.smithi031.stdout:5291 writesame 0x8e1d1fe thru 0x8e2416b data_size 0x22(0x6f6e bytes) 2024-05-08T08:26:05.566 INFO:teuthology.orchestra.run.smithi031.stdout:5293 read 0x948e75f thru 0x9496964 (0x8206 bytes) 2024-05-08T08:26:05.566 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.566 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.566 INFO:teuthology.orchestra.run.smithi031.stdout:5296 write 0x3595bd8 thru 0x359e067 (0x8490 bytes) 2024-05-08T08:26:05.570 INFO:teuthology.orchestra.run.smithi031.stdout:5299 trunc from 0xced8c83 to 0xc4db21b 2024-05-08T08:26:05.575 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:05.575 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:05.577 INFO:teuthology.orchestra.run.smithi031.stdout:5301 write 0xc0d4dcc thru 0xc0d6924 (0x1b59 bytes) 2024-05-08T08:26:05.580 INFO:teuthology.orchestra.run.smithi031.stdout:5302 writesame 0x7c5d6ff thru 0x7c6b540 data_size 0x1d(0xde42 bytes) 2024-05-08T08:26:05.587 INFO:teuthology.orchestra.run.smithi031.stdout:5303 read 0x2be28ac thru 0x2be3788 (0xedd bytes) 2024-05-08T08:26:05.587 INFO:teuthology.orchestra.run.smithi031.stdout:5305 write 0x10292f6 thru 0x1033c17 (0xa922 bytes) 2024-05-08T08:26:05.591 INFO:teuthology.orchestra.run.smithi031.stdout:5306 write 0x79e7e3c thru 0x79f6902 (0xeac7 bytes) 2024-05-08T08:26:05.594 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:05.594 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:05.596 INFO:teuthology.orchestra.run.smithi031.stdout:5307 write 0x8a943af thru 0x8a9ca12 (0x8664 bytes) 2024-05-08T08:26:05.601 INFO:teuthology.orchestra.run.smithi031.stdout:5309 punch from 0x7e8b19b to 0x7e9649b, (0xb300 bytes) 2024-05-08T08:26:05.601 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.601 INFO:teuthology.orchestra.run.smithi031.stdout:5312 punch from 0x796a2cf to 0x797522d, (0xaf5e bytes) 2024-05-08T08:26:05.601 INFO:teuthology.orchestra.run.smithi031.stdout:5314 read 0x829c048 thru 0x82a7eee (0xbea7 bytes) 2024-05-08T08:26:05.602 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.602 INFO:teuthology.orchestra.run.smithi031.stdout:5316 read 0x971193c thru 0x97137ad (0x1e72 bytes) 2024-05-08T08:26:05.602 INFO:teuthology.orchestra.run.smithi031.stdout:5317 read 0x1ef7790 thru 0x1f048ce (0xd13f bytes) 2024-05-08T08:26:05.602 INFO:teuthology.orchestra.run.smithi031.stdout:5320 writesame 0x90efe06 thru 0x90f2350 data_size 0x1(0x254b bytes) 2024-05-08T08:26:05.609 INFO:teuthology.orchestra.run.smithi031.stdout:5321 write 0x9ca42d thru 0x9da145 (0xfd19 bytes) 2024-05-08T08:26:05.613 INFO:teuthology.orchestra.run.smithi031.stdout:5322 read 0x3efa8f3 thru 0x3efced4 (0x25e2 bytes) 2024-05-08T08:26:05.614 INFO:teuthology.orchestra.run.smithi031.stdout:5323 trunc from 0xc4db21b to 0xd6f9069 2024-05-08T08:26:05.616 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:05.617 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:05.620 INFO:teuthology.orchestra.run.smithi031.stdout:5325 read 0x29f16bd thru 0x29fca7d (0xb3c1 bytes) 2024-05-08T08:26:05.620 INFO:teuthology.orchestra.run.smithi031.stdout:5326 read 0x3cac101 thru 0x3cb66fd (0xa5fd bytes) 2024-05-08T08:26:05.620 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.620 INFO:teuthology.orchestra.run.smithi031.stdout:5328 punch from 0xc90da65 to 0xc9169d4, (0x8f6f bytes) 2024-05-08T08:26:05.620 INFO:teuthology.orchestra.run.smithi031.stdout:5329 write 0x1351765 thru 0x135fb20 (0xe3bc bytes) 2024-05-08T08:26:05.622 INFO:teuthology.orchestra.run.smithi031.stdout:5330 read 0x716af60 thru 0x717426a (0x930b bytes) 2024-05-08T08:26:05.622 INFO:teuthology.orchestra.run.smithi031.stdout:5331 writesame 0xcdac8a2 thru 0xcdbae9c data_size 0x19(0xe5fb bytes) 2024-05-08T08:26:05.623 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:05.630+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:05.626 INFO:teuthology.orchestra.run.smithi031.stdout:5333 read 0x86de354 thru 0x86e84d6 (0xa183 bytes) 2024-05-08T08:26:05.626 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.626 INFO:teuthology.orchestra.run.smithi031.stdout:5338 punch from 0x25e4a48 to 0x25f0701, (0xbcb9 bytes) 2024-05-08T08:26:05.626 INFO:teuthology.orchestra.run.smithi031.stdout:5340 read 0xc3c184f thru 0xc3ce26e (0xca20 bytes) 2024-05-08T08:26:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:5343 read 0x9b45f8d thru 0x9b55f43 (0xffb7 bytes) 2024-05-08T08:26:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:5344 read 0x1af82ec thru 0x1b02cb2 (0xa9c7 bytes) 2024-05-08T08:26:05.627 INFO:teuthology.orchestra.run.smithi031.stdout:5345 write 0xa067214 thru 0xa0764d8 (0xf2c5 bytes) 2024-05-08T08:26:05.643 INFO:teuthology.orchestra.run.smithi031.stdout:5346 trunc from 0xd6f9069 to 0x1a0c6c2 2024-05-08T08:26:05.652 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:05.652 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:05.654 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54375 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 933888~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500883780000001, 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060789163, 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54375 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160080+0000", 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160082+0000", 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.35e-06 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160085+0000", 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8430000000000001e-06 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160088+0000", 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.332e-06 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220737+0000", 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060648805 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220808+0000", 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1247000000000004e-05 2024-05-08T08:26:05.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220869+0000", 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1377999999999994e-05 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500872813000001, 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:26:05.659 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:26:05.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500862728, 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500852248999999, 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:26:05.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:05.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:26:05.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500835323, 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:26:05.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:26:05.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500814403, 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:26:05.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:05.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500776088, 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:26:05.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:26:05.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500752342, 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.688 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500729499, 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.50070511, 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.50068319, 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:05.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:26:05.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:26:05.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500661956, 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:26:05.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.500640512, 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:05.709 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.499656207999999, 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.499635007, 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.499623537, 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:26:05.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:26:05.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.499613777, 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.722 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.499603988, 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.726 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:05.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:26:05.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:26:05.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.499593464, 2024-05-08T08:26:05.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:26:05.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:26:05.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.499583784, 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:05.734 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:05.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:26:05.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:26:05.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:05.738 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:05.739 INFO:teuthology.orchestra.run.smithi031.stdout:5347 read 0xa8823 thru 0xb4d34 (0xc512 bytes) 2024-05-08T08:26:05.740 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:05.742 INFO:teuthology.orchestra.run.smithi031.stdout:5349 writesame 0x5affc22 thru 0x5b03cb0 data_size 0x7(0x408f bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5350 punch from 0x1e10d8a to 0x1e16c74, (0x5eea bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5351 punch from 0x220b80 to 0x2268c5, (0x5d45 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5353 read 0x2bf4aec thru 0x2bfbe44 (0x7359 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5354 trunc from 0x5b03cb1 to 0xa288f7b 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5356 punch from 0x99da06d to 0x99e034e, (0x62e1 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5357 write 0x9c3596d thru 0x9c381b1 (0x2845 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5358 read 0x62f8cf9 thru 0x62fb629 (0x2931 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5359 write 0x5ea0da0 thru 0x5eac295 (0xb4f6 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5361 punch from 0x8f3a1f8 to 0x8f496c1, (0xf4c9 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5362 write 0xcd95163 thru 0xcd9b8a2 (0x6740 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5363 writesame 0xb8bfa76 thru 0xb8c49b9 data_size 0x39(0x4f44 bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5364 writesame 0xe0b255b thru 0xe0b5109 data_size 0x1(0x2baf bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:5366 read 0x6d12ad0 thru 0x6d1775c (0x4c8d bytes) 2024-05-08T08:26:05.743 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:5369 writesame 0x4363232 thru 0x436c1b3 data_size 0x4e(0x8f82 bytes) 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:5371 read 0x5d1abc7 thru 0x5d21b74 (0x6fae bytes) 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:5372 writesame 0xe30b87c thru 0xe3111f2 data_size 0x1(0x5977 bytes) 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:5373 write 0x5431578 thru 0x54318cf (0x358 bytes) 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:5375 read 0xb7aa172 thru 0xb7b3752 (0x95e1 bytes) 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:5377 write 0x6f277a9 thru 0x6f2a60f (0x2e67 bytes) 2024-05-08T08:26:05.744 INFO:teuthology.orchestra.run.smithi031.stdout:5380 write 0xcd4e3bd thru 0xcd5235b (0x3f9f bytes) 2024-05-08T08:26:05.744 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:05.747+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:05.745 INFO:teuthology.orchestra.run.smithi031.stdout:5381 read 0x5febcbb thru 0x5ffa55d (0xe8a3 bytes) 2024-05-08T08:26:05.745 INFO:teuthology.orchestra.run.smithi031.stdout:5382 write 0xb316159 thru 0xb3259f3 (0xf89b bytes) 2024-05-08T08:26:05.747 INFO:teuthology.orchestra.run.smithi031.stdout:5384 punch from 0x4f4e1d0 to 0x4f59a11, (0xb841 bytes) 2024-05-08T08:26:05.747 INFO:teuthology.orchestra.run.smithi031.stdout:5385 write 0x819e4a1 thru 0x81a34b3 (0x5013 bytes) 2024-05-08T08:26:05.751 INFO:teuthology.orchestra.run.smithi031.stdout:5386 writesame 0xcec33ec thru 0xcece694 data_size 0x1(0xb2a9 bytes) 2024-05-08T08:26:05.774 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.774 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.786 INFO:teuthology.orchestra.run.smithi031.stdout:5389 trunc from 0xe3111f3 to 0x32b1ce6 2024-05-08T08:26:05.801 INFO:teuthology.orchestra.run.smithi031.stdout:5390 write 0xded350 thru 0xdf0bc8 (0x3879 bytes) 2024-05-08T08:26:05.805 INFO:teuthology.orchestra.run.smithi031.stdout:5391 punch from 0x2ce4ac1 to 0x2cf467a, (0xfbb9 bytes) 2024-05-08T08:26:05.805 INFO:teuthology.orchestra.run.smithi031.stdout:5392 punch from 0x1a9ac74 to 0x1aa0ade, (0x5e6a bytes) 2024-05-08T08:26:05.805 INFO:teuthology.orchestra.run.smithi031.stdout:5395 punch from 0x1e3214 to 0x1e4f68, (0x1d54 bytes) 2024-05-08T08:26:05.812 INFO:teuthology.orchestra.run.smithi031.stdout:5396 write 0x4ac143c thru 0x4acc842 (0xb407 bytes) 2024-05-08T08:26:05.816 INFO:teuthology.orchestra.run.smithi031.stdout:5397 read 0xe099a2 thru 0xe0a78b (0xdea bytes) 2024-05-08T08:26:05.818 INFO:teuthology.orchestra.run.smithi031.stdout:5398 trunc from 0x4acc843 to 0x2d9dcca 2024-05-08T08:26:05.829 INFO:teuthology.orchestra.run.smithi031.stdout:5399 writesame 0x1afa3dd thru 0x1b039dc data_size 0xa0(0x9600 bytes) 2024-05-08T08:26:05.840 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:05.848+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:05.842 INFO:teuthology.orchestra.run.smithi031.stdout:5400 writesame 0x4cbd129 thru 0x4cca009 data_size 0x1(0xcee1 bytes) 2024-05-08T08:26:05.842 INFO:teuthology.orchestra.run.smithi031.stdout:5402 read 0x38407fc thru 0x3845b41 (0x5346 bytes) 2024-05-08T08:26:05.852 INFO:teuthology.orchestra.run.smithi031.stdout:5403 writesame 0x6fa1281 thru 0x6fb0c2a data_size 0x1(0xf9aa bytes) 2024-05-08T08:26:05.852 INFO:teuthology.orchestra.run.smithi031.stdout:5404 writesame 0x5267343 thru 0x526f2f2 data_size 0x48(0x7fb0 bytes) 2024-05-08T08:26:05.855 INFO:teuthology.orchestra.run.smithi031.stdout:5405 read 0x3459d7d thru 0x345db39 (0x3dbd bytes) 2024-05-08T08:26:05.859 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:05.860 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:05.860 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:05.860 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:05.869 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:05.871 INFO:teuthology.orchestra.run.smithi031.stdout:5406 write 0xea3a315 thru 0xea497be (0xf4aa bytes) 2024-05-08T08:26:05.875 INFO:teuthology.orchestra.run.smithi031.stdout:5407 writesame 0x202229e thru 0x202c5ec data_size 0x1(0xa34f bytes) 2024-05-08T08:26:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:05.896 INFO:teuthology.orchestra.run.smithi031.stdout:5409 read 0x9ea97ba thru 0x9eb71c7 (0xda0e bytes) 2024-05-08T08:26:05.903 INFO:teuthology.orchestra.run.smithi031.stdout:5410 write 0xec1bff6 thru 0xec25bc7 (0x9bd2 bytes) 2024-05-08T08:26:05.907 INFO:teuthology.orchestra.run.smithi031.stdout:5412 write 0xc9ad5f6 thru 0xc9bcdc5 (0xf7d0 bytes) 2024-05-08T08:26:05.910 INFO:teuthology.orchestra.run.smithi031.stdout:5413 clone 30 order 25 su 2097152 sc 2 2024-05-08T08:26:05.953 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:05.961+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:06.015 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:06.015 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:06.015 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:06.015 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:06.015 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:06.024 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:06.066 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:06.073+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:06.143 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:06.150+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:7160 3.1a 3:5c311322:::rbd_data.119120fdbc10.0000000000000003:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 49.907096289000002, 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023408063999999999, 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.172 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 7160 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271377+0000", 2024-05-08T08:26:06.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271378+0000", 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.85e-07 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271382+0000", 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1799999999999998e-06 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271392+0000", 2024-05-08T08:26:06.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4790000000000002e-06 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271397+0000", 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0490000000000003e-06 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271447+0000", 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0557999999999999e-05 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.271497+0000", 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9750000000000003e-05 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.275215+0000", 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0037178630000000001 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294610+0000", 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.019395590000000001 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294671+0000", 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0760000000000001e-05 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294729+0000", 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7593999999999998e-05 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:16.294785+0000", 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6456000000000003e-05 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:14789 3.9 3:90dc9cea:::rbd_data.119174045df.00000000000000cf:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:26:06.177 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 40.721998237999998, 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.024649007000000001, 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 14789 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456475+0000", 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456476+0000", 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0279999999999999e-06 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456480+0000", 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1280000000000001e-06 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456491+0000", 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0436e-05 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456497+0000", 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.64e-06 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456545+0000", 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7432999999999998e-05 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.456635+0000", 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9859999999999997e-05 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.460437+0000", 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0038026510000000002 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481041+0000", 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020603652 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481053+0000", 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2301e-05 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481088+0000", 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4669999999999998e-05 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:25.481124+0000", 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6208000000000003e-05 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23289 3.3 3:c7c2a804:::rbd_data.1191ea0f1b1f.0000000000000007:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 35.658856427000003, 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.028538101999999999, 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23289 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519617+0000", 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.519618+0000", 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3170000000000001e-06 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522813+0000", 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0031947130000000001 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522829+0000", 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6005e-05 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522837+0000", 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8990000000000001e-06 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.522902+0000", 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4939999999999998e-05 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523060+0000", 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000158085 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 0", 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.523351+0000", 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00029110399999999998 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.533281+0000", 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099297550000000002 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548076+0000", 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.014795605999999999 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548119+0000", 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2993e-05 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:30.548155+0000", 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5685000000000001e-05 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:23675 3.9 3:93098200:::rbd_data.1191ea0f1b1f.000000000000000b:head [set-alloc-hint object_size 16777216 write_size 16777216,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 35.173897042, 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.027687956, 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 23675 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004576+0000", 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004578+0000", 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.198e-06 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004583+0000", 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5740000000000003e-06 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004593+0000", 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0127e-05 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004599+0000", 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004650+0000", 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0769999999999997e-05 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.004699+0000", 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9186e-05 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 2", 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.009797+0000", 2024-05-08T08:26:06.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0050987319999999999 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032181+0000", 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022383157000000001 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032193+0000", 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2561999999999999e-05 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032228+0000", 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4332000000000003e-05 2024-05-08T08:26:06.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:31.032264+0000", 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6797e-05 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:32844 3.1a 3:5c5225a1:::rbd_object_map.11915c46bf0c:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 19.021402203000001, 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0015158669999999, 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.193 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 32844 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157073+0000", 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157079+0000", 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157093+0000", 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4107999999999999e-05 2024-05-08T08:26:06.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157102+0000", 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0049999999999993e-06 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157167+0000", 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4935000000000001e-05 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157178+0000", 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:26:06.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158227+0000", 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.001049232 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158355+0000", 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012803000000000001 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158587+0000", 2024-05-08T08:26:06.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023214899999999999 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49774 3.3 3:c428cbb1:::rbd_data.11914ad0077c.0000000000000004:1a [read 256000~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.288751304, 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021265240000000001, 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.198 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49774 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889722+0000", 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0600000000000001e-07 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889724+0000", 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8980000000000001e-06 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889730+0000", 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.889735+0000", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7129999999999996e-06 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910823+0000", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021087864000000001 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910898+0000", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5555000000000004e-05 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.910987+0000", 2024-05-08T08:26:06.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9196000000000005e-05 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49825 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 187392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287264706, 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02117349, 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.202 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49825 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891209+0000", 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891211+0000", 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1019999999999999e-06 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891218+0000", 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8569999999999997e-06 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891223+0000", 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.741e-06 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912215+0000", 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020992055999999999 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912288+0000", 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3326999999999998e-05 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912382+0000", 2024-05-08T08:26:06.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3980999999999993e-05 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49826 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 196608~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287244549, 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021377759, 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.206 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49826 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891229+0000", 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891231+0000", 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.7540000000000001e-06 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891237+0000", 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7030000000000003e-06 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891242+0000", 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0329999999999999e-06 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912410+0000", 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021168761000000001 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912484+0000", 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3837999999999993e-05 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912607+0000", 2024-05-08T08:26:06.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012231000000000001 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49827 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 205824~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287225625, 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021638398999999999, 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49827 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891248+0000", 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891250+0000", 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9130000000000001e-06 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891256+0000", 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6570000000000002e-06 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891260+0000", 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5959999999999997e-06 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912651+0000", 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021390180000000002 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912727+0000", 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6477000000000007e-05 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.912886+0000", 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000159176 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49828 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 215040~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287205988, 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021996597, 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.214 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49828 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891267+0000", 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891268+0000", 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891270+0000", 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3640000000000002e-06 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891276+0000", 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6019999999999999e-06 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891281+0000", 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7990000000000001e-06 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913075+0000", 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021794052000000001 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913159+0000", 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4345999999999995e-05 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913264+0000", 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000105071 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49829 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 224256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287186675999999, 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022644764000000001, 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.218 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49829 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891287+0000", 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891289+0000", 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9439999999999999e-06 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891295+0000", 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.896e-06 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891300+0000", 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8350000000000003e-06 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913295+0000", 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021995627 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913372+0000", 2024-05-08T08:26:06.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6736000000000005e-05 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913931+0000", 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00055940299999999998 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49830 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 233472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:06.222 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287167753, 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022779863000000001, 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49830 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:06.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891306+0000", 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8299999999999998e-07 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891308+0000", 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9319999999999999e-06 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891313+0000", 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.31e-06 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891318+0000", 2024-05-08T08:26:06.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5700000000000003e-06 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.913980+0000", 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.02266257 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914025+0000", 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4780999999999997e-05 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914085+0000", 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0316999999999999e-05 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49831 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 242688~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287149849, 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022998424999999999, 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49831 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891324+0000", 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891326+0000", 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.951e-06 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891332+0000", 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8200000000000002e-06 2024-05-08T08:26:06.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891336+0000", 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6789999999999996e-06 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914125+0000", 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022788283999999999 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914206+0000", 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1599000000000003e-05 2024-05-08T08:26:06.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914322+0000", 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00011571299999999999 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49832 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 251904~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287131709000001, 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023170733999999998, 2024-05-08T08:26:06.231 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49832 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891342+0000", 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891344+0000", 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891350+0000", 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5679999999999999e-06 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891354+0000", 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5750000000000002e-06 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914351+0000", 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.022996632999999999 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914423+0000", 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2469e-05 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914512+0000", 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9166999999999995e-05 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49833 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 261120~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:06.235 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.287113629, 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023398988999999999, 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49833 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891360+0000", 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891362+0000", 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8169999999999999e-06 2024-05-08T08:26:06.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891367+0000", 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5110000000000003e-06 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.891373+0000", 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5890000000000002e-06 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914541+0000", 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.023167982 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914614+0000", 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3317000000000003e-05 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914759+0000", 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00014440500000000001 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.239 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50084 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 216064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.279530457, 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021113683000000001, 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.240 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.241 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50084 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898943+0000", 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3700000000000001e-07 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898945+0000", 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9520000000000001e-06 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898951+0000", 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2850000000000002e-06 2024-05-08T08:26:06.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.898955+0000", 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8629999999999999e-06 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919894+0000", 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020938667000000001 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919965+0000", 2024-05-08T08:26:06.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0927999999999996e-05 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920057+0000", 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1650999999999994e-05 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50085 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 225280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.279272373, 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021069139000000001, 2024-05-08T08:26:06.245 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50085 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899201+0000", 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899202+0000", 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0999999999999997e-07 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899206+0000", 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.861e-06 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899214+0000", 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1289999999999996e-06 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899220+0000", 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5749999999999997e-06 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920083+0000", 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.020862975999999998 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920177+0000", 2024-05-08T08:26:06.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3540000000000002e-05 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920270+0000", 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3247999999999999e-05 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50086 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 234496~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.279236214999999, 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021223872000000001, 2024-05-08T08:26:06.249 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50086 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899237+0000", 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899238+0000", 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1099999999999995e-07 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899241+0000", 2024-05-08T08:26:06.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8100000000000002e-06 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899247+0000", 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4139999999999999e-06 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899252+0000", 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7389999999999999e-06 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920299+0000", 2024-05-08T08:26:06.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021046914999999999 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920373+0000", 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3962999999999996e-05 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920461+0000", 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.8419999999999997e-05 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50087 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 243712~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.279214128, 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021404211999999999, 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50087 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899259+0000", 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899260+0000", 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0600000000000001e-07 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899262+0000", 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.058e-06 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899267+0000", 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6699999999999999e-06 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899272+0000", 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6920000000000001e-06 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920490+0000", 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021217881000000001 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920561+0000", 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1377000000000005e-05 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920663+0000", 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102128 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.257 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:50088 3.1a 3:5af5de94:::rbd_data.11914ad0077c.0000000000000006:1a [read 252928~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.279193885, 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021585928000000001, 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.258 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 50088 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899279+0000", 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899280+0000", 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899282+0000", 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1330000000000002e-06 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899288+0000", 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6890000000000001e-06 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.899293+0000", 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8160000000000001e-06 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920704+0000", 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.021411660999999999 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920778+0000", 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3788999999999994e-05 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.920865+0000", 2024-05-08T08:26:06.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7437999999999999e-05 2024-05-08T08:26:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.262 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.262 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:06.263 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:26:06.265 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:06.265 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:06.271+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:06.365 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:06.372+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:06.411 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:06.411 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:06.411 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:06.412 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:06.420 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:06.458 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-05-08T08:26:06.459 INFO:teuthology.orchestra.run.smithi031.stdout:5414 punch from 0x126a810 to 0x1275080, (0xa870 bytes) 2024-05-08T08:26:06.462 INFO:teuthology.orchestra.run.smithi031.stdout:5415 punch from 0x9e4 to 0xc522, (0xbb3e bytes) 2024-05-08T08:26:06.463 INFO:teuthology.orchestra.run.smithi031.stdout:5416 read 0x66d8802 thru 0x66dc66a (0x3e69 bytes) 2024-05-08T08:26:06.463 INFO:teuthology.orchestra.run.smithi031.stdout:5418 read 0xb904f17 thru 0xb90aea7 (0x5f91 bytes) 2024-05-08T08:26:06.463 INFO:teuthology.orchestra.run.smithi031.stdout:5419 read 0x5800005 thru 0x5806bc5 (0x6bc1 bytes) 2024-05-08T08:26:06.463 INFO:teuthology.orchestra.run.smithi031.stdout:5420 write 0x56060ae thru 0x561402c (0xdf7f bytes) 2024-05-08T08:26:06.466 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:06.473+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:06.479 INFO:teuthology.orchestra.run.smithi031.stdout:5421 trunc from 0xec25bc8 to 0x4486374 2024-05-08T08:26:06.489 INFO:teuthology.orchestra.run.smithi031.stdout:5422 read 0x2a54ddf thru 0x2a5df1f (0x9141 bytes) 2024-05-08T08:26:06.489 INFO:teuthology.orchestra.run.smithi031.stdout:5423 write 0x31ba69a thru 0x31be357 (0x3cbe bytes) 2024-05-08T08:26:06.508 INFO:teuthology.orchestra.run.smithi031.stdout:5424 writesame 0x8f2c2c1 thru 0x8f38244 data_size 0xe(0xbf84 bytes) 2024-05-08T08:26:06.509 INFO:teuthology.orchestra.run.smithi031.stdout:5425 punch from 0x3ae2de4 to 0x3ae8114, (0x5330 bytes) 2024-05-08T08:26:06.510 INFO:teuthology.orchestra.run.smithi031.stdout:5429 trunc from 0x8f38245 to 0x7912835 2024-05-08T08:26:06.529 INFO:teuthology.orchestra.run.smithi031.stdout:5430 write 0xdc7ec2d thru 0xdc84e6f (0x6243 bytes) 2024-05-08T08:26:06.533 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:06.533 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:06.533 INFO:teuthology.orchestra.run.smithi031.stdout:5432 punch from 0x2d048bd to 0x2d10a77, (0xc1ba bytes) 2024-05-08T08:26:06.542 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:06.546 INFO:teuthology.orchestra.run.smithi031.stdout:5434 trunc from 0xdc84e70 to 0x219f9e5 2024-05-08T08:26:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:5435 read 0x1b280ba thru 0x1b3381c (0xb763 bytes) 2024-05-08T08:26:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:5436 read 0x33ca97 thru 0x33e5c0 (0x1b2a bytes) 2024-05-08T08:26:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:5437 punch from 0x1acd14 to 0x1b7fc0, (0xb2ac bytes) 2024-05-08T08:26:06.563 INFO:teuthology.orchestra.run.smithi031.stdout:5438 read 0x2eec4e thru 0x2f5578 (0x692b bytes) 2024-05-08T08:26:06.566 INFO:teuthology.orchestra.run.smithi031.stdout:5439 trunc from 0x219f9e5 to 0x3f05f28 2024-05-08T08:26:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:5440 punch from 0x2cf590a to 0x2cfb0bf, (0x57b5 bytes) 2024-05-08T08:26:06.571 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:06.578 INFO:teuthology.orchestra.run.smithi031.stdout:5443 writesame 0x56d77aa thru 0x56dec4b data_size 0x1(0x74a2 bytes) 2024-05-08T08:26:06.578 INFO:teuthology.orchestra.run.smithi031.stdout:5444 read 0x1c71892 thru 0x1c753bd (0x3b2c bytes) 2024-05-08T08:26:06.579 INFO:teuthology.orchestra.run.smithi031.stdout:5445 punch from 0x43a0b05 to 0x43a19e3, (0xede bytes) 2024-05-08T08:26:06.583 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:06.591+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:06.592 INFO:teuthology.orchestra.run.smithi031.stdout:5448 writesame 0xc0e9933 thru 0xc0f7829 data_size 0x1(0xdef7 bytes) 2024-05-08T08:26:06.597 INFO:teuthology.orchestra.run.smithi031.stdout:5449 write 0xc27221f thru 0xc27ad03 (0x8ae5 bytes) 2024-05-08T08:26:06.601 INFO:teuthology.orchestra.run.smithi031.stdout:5450 read 0x50115a0 thru 0x501e18c (0xcbed bytes) 2024-05-08T08:26:06.601 INFO:teuthology.orchestra.run.smithi031.stdout:5451 write 0x6b28a1 thru 0x6c09c5 (0xe125 bytes) 2024-05-08T08:26:06.603 INFO:teuthology.orchestra.run.smithi031.stdout:5452 writesame 0x4fb85f8 thru 0x4fbf13c data_size 0x1(0x6b45 bytes) 2024-05-08T08:26:06.618 INFO:teuthology.orchestra.run.smithi031.stdout:5453 writesame 0x46f626 thru 0x47b25f data_size 0x9(0xbc3a bytes) 2024-05-08T08:26:06.621 INFO:teuthology.orchestra.run.smithi031.stdout:5454 write 0x99c2833 thru 0x99c6a1b (0x41e9 bytes) 2024-05-08T08:26:06.626 INFO:teuthology.orchestra.run.smithi031.stdout:5455 read 0x509c06c thru 0x50a4e20 (0x8db5 bytes) 2024-05-08T08:26:06.634 INFO:teuthology.orchestra.run.smithi031.stdout:5456 trunc from 0xc27ad04 to 0x52e28f9 2024-05-08T08:26:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:06.645 INFO:teuthology.orchestra.run.smithi031.stdout:5461 punch from 0x2309718 to 0x230d591, (0x3e79 bytes) 2024-05-08T08:26:06.655 INFO:teuthology.orchestra.run.smithi031.stdout:5462 writesame 0x87bd0d8 thru 0x87bfc2d data_size 0x2b(0x2b56 bytes) 2024-05-08T08:26:06.655 INFO:teuthology.orchestra.run.smithi031.stdout:5463 clone 31 order 25 su 8388608 sc 14 2024-05-08T08:26:06.667 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:06.675+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:06.693 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:06.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.039504053000002, 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:06.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.699 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.786023031999999, 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.703 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.785971084, 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.785936229000001, 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.785905965, 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:06.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:06.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.715 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.784596599, 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:06.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.719 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.784575837, 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:06.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.784504367, 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.727 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.784476999000001, 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:06.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.731 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.782522609000001, 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.732 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.782503042, 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:06.739 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.782483272, 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.740 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:06.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:06.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.782464274000001, 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.744 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:06.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:06.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.780564638, 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.748 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.780544670999999, 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:06.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.780523098, 2024-05-08T08:26:06.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.780399107999999, 2024-05-08T08:26:06.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.780370313000001, 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:06.764 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.778452776, 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:06.768 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.778431321999999, 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:06.772 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:06.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:06.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:06.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:06.776 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:06.777 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:06.778 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:06.785+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:06.878 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:06.886+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:06.925 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:06.926 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:06.934 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:06.979 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:06.987+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:07.039 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:07.039 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:07.039 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:07.039 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:07.039 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:07.048 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:07.080 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:07.088+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:07.182 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:07.190+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4241459369999996, 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.198 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:07.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:07.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:07.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4241252190000004, 2024-05-08T08:26:07.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.424097282, 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:07.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:07.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.210 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4240755189999996, 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:07.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:07.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:07.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4240529410000002, 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:07.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4240128719999996, 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.222 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4239905640000003, 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.223 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:07.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:07.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4239619829999999, 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4239377300000005, 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.231 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4239053320000004, 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.235 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:07.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:07.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.422567527, 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4225455089999999, 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4225031499999998, 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.247 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:07.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:07.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:07.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.422461631, 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:07.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:07.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:07.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:07.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4204835349999998, 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4204722409999997, 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4163844640000001, 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4163689870000002, 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4112502119999997, 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.271 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:07.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:07.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.4109722829999995, 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.275 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:07.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:07.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.279 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:07.280 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:26:07.281 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:07.322 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:07.330+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:07.328 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone30 from 0x87bfc2e (overlap 0x219f9e5) to 0xd730c2 2024-05-08T08:26:07.383 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:07.391+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:07.389 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-05-08T08:26:07.430 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:07.430 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:07.439 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:07.484 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:07.492+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:07.546 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:07.546 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:07.546 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:07.546 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:07.546 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:07.554 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:07.585 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:07.593+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:07.648 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:07.648 INFO:teuthology.orchestra.run.smithi031.stdout:5465 writesame 0x6a338ab thru 0x6a3e023 data_size 0x3(0xa779 bytes) 2024-05-08T08:26:07.662 INFO:teuthology.orchestra.run.smithi031.stdout:5467 read 0x56ee694 thru 0x56fce91 (0xe7fe bytes) 2024-05-08T08:26:07.662 INFO:teuthology.orchestra.run.smithi031.stdout:5468 read 0x37a7867 thru 0x37b62f0 (0xea8a bytes) 2024-05-08T08:26:07.662 INFO:teuthology.orchestra.run.smithi031.stdout:5469 read 0x42df5f5 thru 0x42edd9b (0xe7a7 bytes) 2024-05-08T08:26:07.663 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:07.663 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:07.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 70.104492305999997, 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.664 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:07.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:07.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:07.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8803869989999997, 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.670 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.880301845, 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.674 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8802739390000003, 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.678 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:07.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8802455380000005, 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.682 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8801613990000003, 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.686 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:07.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:07.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8800809039999997, 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.690 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:07.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8800483239999997, 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.694 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8799951329999995, 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.698 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8799691540000003, 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.702 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8799138219999998, 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.706 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8797637890000001, 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.710 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:07.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:07.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.87971413, 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.714 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:07.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.717 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8796795609999997, 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.718 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8796575539999996, 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.722 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:07.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:07.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8796345800000003, 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.726 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8796049019999996, 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.730 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:07.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:07.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8783823240000004, 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.734 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:07.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8783718299999999, 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.738 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:07.742 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 5.8732529939999996, 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:07.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:07.746 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:07.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:07.749 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:07.750 INFO:teuthology.orchestra.run.smithi031.stdout:5471 write 0x3dc6a3b thru 0x3dcd30f (0x68d5 bytes) 2024-05-08T08:26:07.750 INFO:teuthology.orchestra.run.smithi031.stdout:5472 read 0x462a554 thru 0x462ef10 (0x49bd bytes) 2024-05-08T08:26:07.750 INFO:teuthology.orchestra.run.smithi031.stdout:5474 punch from 0x1d6ebc3 to 0x1d77bbb, (0x8ff8 bytes) 2024-05-08T08:26:07.750 INFO:teuthology.orchestra.run.smithi031.stdout:5475 flatten 2024-05-08T08:26:07.750 INFO:teuthology.orchestra.run.smithi031.stdout:5476 read 0x104f900 thru 0x1054f03 (0x5604 bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5477 punch from 0x46a7c52 to 0x46ac07f, (0x442d bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5478 write 0xe1fd13a thru 0xe20271a (0x55e1 bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5479 read 0x75cc97a thru 0x75dc354 (0xf9db bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5480 read 0x57623d8 thru 0x576715d (0x4d86 bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5481 write 0x5045a79 thru 0x504f279 (0x9801 bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5483 writesame 0x70052e3 thru 0x7013380 data_size 0x1(0xe09e bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5484 read 0xbb0595 thru 0xbb9bb5 (0x9621 bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5485 read 0x15394c7 thru 0x1548dff (0xf939 bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5488 punch from 0x159622e to 0x15a3ba9, (0xd97b bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5489 write 0xc770ace thru 0xc778909 (0x7e3c bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5491 write 0x5f59522 thru 0x5f62c2b (0x970a bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5492 write 0xe777617 thru 0xe77b8f5 (0x42df bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5493 read 0x222cc19 thru 0x2237d9d (0xb185 bytes) 2024-05-08T08:26:07.751 INFO:teuthology.orchestra.run.smithi031.stdout:5494 punch from 0xcf239b4 to 0xcf2a337, (0x6983 bytes) 2024-05-08T08:26:07.752 INFO:teuthology.orchestra.run.smithi031.stdout:5495 read 0xd34f30 thru 0xd357d6 (0x8a7 bytes) 2024-05-08T08:26:07.752 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:07.752 INFO:teuthology.orchestra.run.smithi031.stdout:5497 clone 32 order 18 su 2048 sc 14 2024-05-08T08:26:07.752 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:07.755+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:07.849 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:07.856+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:07.863 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:07.863 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:07.864 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:07.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:07.879 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:07.880 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:07.898 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:07.899 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:07.917 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:07.918 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:07.935 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:07.935 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:07.949 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:07.957+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:07.953 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:07.953 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:07.971 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:07.971 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:07.989 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:07.989 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:07.991 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:07.991 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:08.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:08.007 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:08.008 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:08.060 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:08.068+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:08.117 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:08.117 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:08.125 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:08.151 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:08.159+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:08.251 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:08.259+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:08.254 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:08.254 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:08.254 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-05-08T08:26:08.254 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T08:26:08.254 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T08:26:08.254 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-05-08T08:26:08.280 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:08.280 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:08.291 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:08.375 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:08.383+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:08.377 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:08.377 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T08:26:08.377 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:08.448 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:08.448 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:08.453 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:08.460+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:08.457 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:08.470 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-05-08T08:26:08.489 INFO:teuthology.orchestra.run.smithi031.stdout:5498 read 0x9980828 thru 0x998e273 (0xda4c bytes) 2024-05-08T08:26:08.490 INFO:teuthology.orchestra.run.smithi031.stdout:5500 read 0x594a8f2 thru 0x5956765 (0xbe74 bytes) 2024-05-08T08:26:08.498 INFO:teuthology.orchestra.run.smithi031.stdout:5503 trunc from 0xe77b8f6 to 0xb9a8116 2024-05-08T08:26:08.554 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:08.562+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:08.597 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:08.598 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:08.608 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:08.692 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:08.699+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:08.756 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:08.764+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:08.761 INFO:teuthology.orchestra.run.smithi031.stdout:5504 trunc from 0xb9a8116 to 0xc5ded1b 2024-05-08T08:26:08.763 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:08.763 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:08.766 INFO:teuthology.orchestra.run.smithi031.stdout:5507 write 0x4dcb47c thru 0x4dd8aff (0xd684 bytes) 2024-05-08T08:26:08.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:08.859 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:08.867+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:08.935 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:08.935 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:08.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:08.958 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:08.966+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:08.998 INFO:teuthology.orchestra.run.smithi031.stdout:5509 write 0x120b775 thru 0x1212158 (0x69e4 bytes) 2024-05-08T08:26:09.060 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:09.066+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:09.105 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:09.105 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:09.115 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:09.198 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:09.206+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:09.255 INFO:teuthology.orchestra.run.smithi031.stdout:5510 write 0x75b4ec2 thru 0x75bf4be (0xa5fd bytes) 2024-05-08T08:26:09.262 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:09.270+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:09.282 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:26:09.295 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:09.295 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:09.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:09.394 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:09.402+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:09.464 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:09.471+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:09.496 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:09.496 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:09.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:09.592 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:09.600+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:09.626 INFO:teuthology.orchestra.run.smithi031.stdout:5511 write 0xe3410e9 thru 0xe344638 (0x3550 bytes) 2024-05-08T08:26:09.639 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:09.639 INFO:teuthology.orchestra.run.smithi031.stdout:5513 writesame 0xa9537dd thru 0xa95cd31 data_size 0x1(0x9555 bytes) 2024-05-08T08:26:09.665 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:09.672+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:09.705 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:09.705 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:09.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:09.766 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:09.773+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:09.824 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:09.824 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:09.833 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:09.874 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:09.881+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:09.917 INFO:teuthology.orchestra.run.smithi031.stdout:5515 writesame 0x4a7307f thru 0x4a82cda data_size 0x3e(0xfc5c bytes) 2024-05-08T08:26:09.968 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:09.975+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:10.014 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:10.014 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:10.020 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:10.024 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:10.069 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:10.076+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:10.161 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:10.169 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:10.176+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:10.180 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:10.180 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:10.198 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:10.198 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:10.199 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:10.199 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:10.203 INFO:teuthology.orchestra.run.smithi031.stdout:5517 punch from 0x6d6c470 to 0x6d78c0c, (0xc79c bytes) 2024-05-08T08:26:10.203 INFO:teuthology.orchestra.run.smithi031.stdout:5518 read 0xb580257 thru 0xb58a51f (0xa2c9 bytes) 2024-05-08T08:26:10.204 INFO:teuthology.orchestra.run.smithi031.stdout:5519 read 0x91ecb99 thru 0x91f15d0 (0x4a38 bytes) 2024-05-08T08:26:10.205 INFO:teuthology.orchestra.run.smithi031.stdout:5521 write 0x6ff4363 thru 0x7003e77 (0xfb15 bytes) 2024-05-08T08:26:10.209 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:10.224 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:10.225 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:10.251 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:10.274 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:10.279 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:26:10.292 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:10.299+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:10.302 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:10.323 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:10.324 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:10.359 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:10.359 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:10.359 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:10.359 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:10.369 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:10.411 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:10.417+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:10.472 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:10.479+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:10.518 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:10.518 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:10.518 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:10.518 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:10.518 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:10.527 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:10.573 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:10.580+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:10.674 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:10.681+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 41.023430519000001, 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.678 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.769949497999999, 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.684 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.76989755, 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:10.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:10.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.769862695, 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.693 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:10.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.697 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.769832431000001, 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.698 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:10.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:10.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:10.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.768523065, 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.702 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.768502302999998, 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.706 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:10.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:10.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.768430833, 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.710 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.768403464999999, 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.714 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:10.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.766449075000001, 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.718 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.722 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.766429508000002, 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:10.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.766409738, 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.727 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:10.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.766390739999999, 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:10.731 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.735 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.764491104000001, 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:10.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:10.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.764471137000001, 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:10.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:10.743 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.764449564, 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:10.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:10.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.764325574000001, 2024-05-08T08:26:10.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:10.752 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.764296778999999, 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:10.756 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.762379242000002, 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.760 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.762357787999999, 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:10.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.764 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.765 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.765 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:10.765 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:10.765 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:10.765 INFO:teuthology.orchestra.run.smithi031.stdout:5522 trunc from 0xe344639 to 0xe92d6d 2024-05-08T08:26:10.766 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:10.775 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:10.782+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:10.875 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:10.883+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:10.892 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:10.892 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:10.892 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:10.892 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:10.901 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:10.950 INFO:teuthology.orchestra.run.smithi031.stdout:5523 writesame 0xa14f968 thru 0xa157190 data_size 0x1(0x7829 bytes) 2024-05-08T08:26:10.950 INFO:teuthology.orchestra.run.smithi031.stdout:5524 write 0x1564767 thru 0x1571a38 (0xd2d2 bytes) 2024-05-08T08:26:10.974 INFO:teuthology.orchestra.run.smithi031.stdout:5525 trunc from 0xa157191 to 0xeab1c6d 2024-05-08T08:26:10.975 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:10.983+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:10.980 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:5527 read 0x8a6dc65 thru 0x8a78c38 (0xafd4 bytes) 2024-05-08T08:26:10.981 INFO:teuthology.orchestra.run.smithi031.stdout:5530 write 0x96253f7 thru 0x9633fc1 (0xebcb bytes) 2024-05-08T08:26:10.996 INFO:teuthology.orchestra.run.smithi031.stdout:5531 writesame 0x48f7e27 thru 0x48fc0d8 data_size 0x1(0x42b2 bytes) 2024-05-08T08:26:11.009 INFO:teuthology.orchestra.run.smithi031.stdout:5532 writesame 0x520acab thru 0x5218e7f data_size 0x7(0xe1d5 bytes) 2024-05-08T08:26:11.021 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:11.021 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:11.021 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:11.021 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:11.021 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:11.025 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:11.031 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:11.036 INFO:teuthology.orchestra.run.smithi031.stdout:5534 trunc from 0xeab1c6d to 0x512cf85 2024-05-08T08:26:11.063 INFO:teuthology.orchestra.run.smithi031.stdout:5536 trunc from 0x512cf85 to 0x123f4ae 2024-05-08T08:26:11.076 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:11.083+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:11.088 INFO:teuthology.orchestra.run.smithi031.stdout:5537 read 0xfc49bd thru 0xfc88f8 (0x3f3c bytes) 2024-05-08T08:26:11.089 INFO:teuthology.orchestra.run.smithi031.stdout:5538 read 0x8405e8 thru 0x84b743 (0xb15c bytes) 2024-05-08T08:26:11.093 INFO:teuthology.orchestra.run.smithi031.stdout:5539 trunc from 0x123f4ae to 0x43323a 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3744756589999998, 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:11.148 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.152 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3744549409999998, 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:11.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.157 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3744270039999993, 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.161 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3744052409999998, 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.162 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.166 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:11.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:11.166 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.166 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.166 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3743826630000004, 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:11.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:11.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3743425939999998, 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.171 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:11.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3743202859999997, 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.175 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:11.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.179 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3742917049999992, 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:11.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:11.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3742674519999998, 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.184 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:11.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:11.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3742350539999997, 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:11.188 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:11.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:11.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.192 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3728972489999993, 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.196 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3728752310000001, 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.197 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.372832872, 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.201 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:11.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:11.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:11.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3727913530000002, 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:11.205 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:11.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.370813257, 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.210 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:11.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3708019629999999, 2024-05-08T08:26:11.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:11.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:11.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:11.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3667141859999994, 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.219 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:11.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:11.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:11.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3666987089999996, 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3615799339999999, 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.231 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.3613020050000006, 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:11.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:11.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:11.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:11.236 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.236 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.236 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.236 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.236 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.236 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:11.237 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:11.238 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:11.244+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:11.338 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:11.345+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:11.386 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:11.386 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:11.386 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:11.386 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:11.395 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:11.432 INFO:teuthology.orchestra.run.smithi031.stdout:5540 writesame 0x1a8c264 thru 0x1a945de data_size 0x1(0x837b bytes) 2024-05-08T08:26:11.439 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:11.445+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:11.446 INFO:teuthology.orchestra.run.smithi031.stdout:5541 write 0x88bdb02 thru 0x88c6e78 (0x9377 bytes) 2024-05-08T08:26:11.462 INFO:teuthology.orchestra.run.smithi031.stdout:5542 write 0x4ef532c thru 0x4efe96c (0x9641 bytes) 2024-05-08T08:26:11.479 INFO:teuthology.orchestra.run.smithi031.stdout:5543 trunc from 0x88c6e79 to 0x6c70b1b 2024-05-08T08:26:11.496 INFO:teuthology.orchestra.run.smithi031.stdout:5544 write 0x6cf7b8a thru 0x6cf9a14 (0x1e8b bytes) 2024-05-08T08:26:11.503 INFO:teuthology.orchestra.run.smithi031.stdout:5545 punch from 0x2ea44a0 to 0x2eadb73, (0x96d3 bytes) 2024-05-08T08:26:11.503 INFO:teuthology.orchestra.run.smithi031.stdout:5546 punch from 0xb174df to 0xb26974, (0xf495 bytes) 2024-05-08T08:26:11.503 INFO:teuthology.orchestra.run.smithi031.stdout:5549 read 0x12a7c28 thru 0x12b45ed (0xc9c6 bytes) 2024-05-08T08:26:11.503 INFO:teuthology.orchestra.run.smithi031.stdout:5551 read 0x489451a thru 0x4899a9a (0x5581 bytes) 2024-05-08T08:26:11.503 INFO:teuthology.orchestra.run.smithi031.stdout:5552 punch from 0x84f325 to 0x84fc89, (0x964 bytes) 2024-05-08T08:26:11.507 INFO:teuthology.orchestra.run.smithi031.stdout:5554 trunc from 0x6cf9a15 to 0x334f1a4 2024-05-08T08:26:11.515 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:11.515 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:11.515 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:11.515 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:11.515 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:11.523 INFO:teuthology.orchestra.run.smithi031.stdout:5555 read 0x298c7af thru 0x298ca3e (0x290 bytes) 2024-05-08T08:26:11.523 INFO:teuthology.orchestra.run.smithi031.stdout:5556 read 0x1710be thru 0x17ebdb (0xdb1e bytes) 2024-05-08T08:26:11.524 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:11.539 INFO:teuthology.orchestra.run.smithi031.stdout:5557 trunc from 0x334f1a4 to 0xd8a5c12 2024-05-08T08:26:11.539 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:11.547+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:5558 read 0x8c0fcc1 thru 0x8c1ed58 (0xf098 bytes) 2024-05-08T08:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:5563 punch from 0x326a42d to 0x3278b8a, (0xe75d bytes) 2024-05-08T08:26:11.545 INFO:teuthology.orchestra.run.smithi031.stdout:5564 write 0x8c69199 thru 0x8c76842 (0xd6aa bytes) 2024-05-08T08:26:11.564 INFO:teuthology.orchestra.run.smithi031.stdout:5567 write 0x77663d3 thru 0x776736e (0xf9c bytes) 2024-05-08T08:26:11.573 INFO:teuthology.orchestra.run.smithi031.stdout:5568 read 0x902c45b thru 0x9038cb3 (0xc859 bytes) 2024-05-08T08:26:11.574 INFO:teuthology.orchestra.run.smithi031.stdout:5570 clone 33 order 21 su 8192 sc 10 2024-05-08T08:26:11.640 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:11.646+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.113142698000004, 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:11.672 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:11.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:11.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:11.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8890373910000005, 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:11.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:11.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8889522369999998, 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:11.683 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:11.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.687 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8889243310000001, 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:11.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:11.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8888959300000003, 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8888117910000002, 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.696 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:11.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.700 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8887312959999996, 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.704 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8886987160000004, 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:11.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:11.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:11.708 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8886455249999994, 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.709 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:11.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:11.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.712 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8886195459999993, 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.713 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:11.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:11.716 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8885642140000005, 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:11.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:11.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:11.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:11.720 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8884141809999999, 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:11.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8883645219999998, 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8883299529999995, 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:11.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:11.732 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8883079459999994, 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8882849719999992, 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.737 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8882552940000004, 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:11.741 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8870327160000002, 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:11.745 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.749 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8870222220000006, 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:11.753 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 9.8819033859999994, 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:11.757 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:11.758 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:11.759 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:11.766+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:11.859 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:11.866+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:11.870 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:11.870 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:11.870 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:11.870 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:11.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:11.960 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:11.968+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:11.988 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:11.988 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:11.988 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:11.988 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:11.988 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:11.996 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:12.080 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:12.088+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:12.148 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50476 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 161792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239712887, 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018840722000000001, 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.149 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50476 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915327+0000", 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915329+0000", 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915336+0000", 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4880000000000001e-06 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915342+0000", 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.933988+0000", 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018646593 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934072+0000", 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3626000000000002e-05 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934167+0000", 2024-05-08T08:26:12.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5087999999999995e-05 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50477 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 171008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239690414999998, 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019055283999999999, 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50477 2024-05-08T08:26:12.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915351+0000", 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.311e-06 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915363+0000", 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2031000000000001e-05 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915369+0000", 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6930000000000004e-06 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934223+0000", 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018853741 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934307+0000", 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4154999999999996e-05 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934404+0000", 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6973999999999997e-05 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50478 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 180224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239661795, 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019257972000000002, 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.157 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50478 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:12.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915378+0000", 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915387+0000", 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999993e-06 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915396+0000", 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2499999999999995e-06 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915401+0000", 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.259e-06 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934434+0000", 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019032970999999999 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934535+0000", 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100479 2024-05-08T08:26:12.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934635+0000", 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100445 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50479 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 189440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239629535999999, 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019428513000000001, 2024-05-08T08:26:12.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50479 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915413+0000", 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915419+0000", 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7320000000000001e-06 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915424+0000", 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934664+0000", 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019239903999999999 2024-05-08T08:26:12.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934740+0000", 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5833000000000004e-05 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934838+0000", 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7832999999999997e-05 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.165 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50480 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 198656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239607625000001, 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019651075, 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50480 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915432+0000", 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915434+0000", 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.221e-06 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915440+0000", 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4770000000000002e-06 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915444+0000", 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8330000000000001e-06 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934880+0000", 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019435555 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934984+0000", 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010403099999999999 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935083+0000", 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8566000000000005e-05 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50481 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239587687, 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019849193000000001, 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50481 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915452+0000", 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915454+0000", 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3719999999999999e-06 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915461+0000", 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6290000000000004e-06 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915466+0000", 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935126+0000", 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019660074 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935221+0000", 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4598000000000005e-05 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935301+0000", 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.996e-05 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.173 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50482 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239566107999998, 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020069704000000001, 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50482 2024-05-08T08:26:12.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915476+0000", 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915482+0000", 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4030000000000004e-06 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915487+0000", 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9219999999999996e-06 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935343+0000", 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019856492999999999 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935433+0000", 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9485000000000001e-05 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935543+0000", 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109832 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50483 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 226304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239545783000001, 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020296027000000001, 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.178 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50483 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:12.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915494+0000", 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915496+0000", 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915502+0000", 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7939999999999999e-06 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915507+0000", 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935582+0000", 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020075790999999999 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935672+0000", 2024-05-08T08:26:12.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0074999999999994e-05 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935789+0000", 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011681299999999999 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50484 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239525923999999, 2024-05-08T08:26:12.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020571613999999998, 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50484 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915514+0000", 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915516+0000", 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2809999999999998e-06 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915649+0000", 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000132776 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915668+0000", 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9253e-05 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935846+0000", 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020177820999999999 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935950+0000", 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104478 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936085+0000", 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000134618 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.186 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50485 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.239359237999999, 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020672174000000001, 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50485 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4700000000000001e-07 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915684+0000", 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5820000000000002e-06 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915692+0000", 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1829999999999994e-06 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915706+0000", 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3747999999999999e-05 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936120+0000", 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020414472999999999 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936205+0000", 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4561000000000006e-05 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936352+0000", 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014698000000000001 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50486 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.23932336, 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020885167, 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50486 2024-05-08T08:26:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915724+0000", 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4200000000000001e-06 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915734+0000", 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0796e-05 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915748+0000", 2024-05-08T08:26:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3318e-05 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936388+0000", 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020640642000000001 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936491+0000", 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010300400000000001 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936601+0000", 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109523 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50737 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 208896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.231370452, 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018673419, 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50737 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923672+0000", 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.252e-06 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923679+0000", 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923737+0000", 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8035000000000003e-05 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942158+0000", 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018421334000000001 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942240+0000", 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1409000000000006e-05 2024-05-08T08:26:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942342+0000", 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102252 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50738 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.231287104, 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018832728, 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50738 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923753+0000", 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-07 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923756+0000", 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4510000000000001e-06 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923766+0000", 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.049e-05 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923773+0000", 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942387+0000", 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018614117999999999 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942475+0000", 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7874999999999995e-05 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942585+0000", 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011014300000000001 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50739 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.23122214, 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990561999999999, 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50739 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923828+0000", 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0988999999999999e-05 2024-05-08T08:26:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923837+0000", 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6889999999999993e-06 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923851+0000", 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3528e-05 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942618+0000", 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018766998999999999 2024-05-08T08:26:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942704+0000", 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5995000000000006e-05 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942807+0000", 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103837 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50740 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.231178755999998, 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019168166, 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50740 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923861+0000", 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923863+0000", 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923871+0000", 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0569999999999993e-06 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923877+0000", 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942837+0000", 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018959999000000002 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942917+0000", 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0519000000000006e-05 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943028+0000", 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111402 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50741 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.231154908000001, 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019309865999999998, 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50741 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923891+0000", 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923894+0000", 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1829999999999998e-06 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923902+0000", 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0910000000000004e-06 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923907+0000", 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6729999999999997e-06 2024-05-08T08:26:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943059+0000", 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019151695 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943123+0000", 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4098999999999997e-05 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943194+0000", 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0970999999999995e-05 2024-05-08T08:26:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50742 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 20.231123984, 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019467485, 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50742 2024-05-08T08:26:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923932+0000", 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6268000000000001e-05 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923939+0000", 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7270000000000007e-06 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923945+0000", 2024-05-08T08:26:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5419999999999996e-06 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943232+0000", 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019287316999999998 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943315+0000", 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2854000000000003e-05 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943383+0000", 2024-05-08T08:26:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7427000000000003e-05 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.994784995, 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.994763894999998, 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.994743237000002, 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:12.232 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:12.234 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:12.234 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:12.241+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:12.300 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:26:12.334 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:12.341+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:12.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:12.352 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone32 from 0xd8a5c12 (overlap 0x43323a) to 0x35c1c8 2024-05-08T08:26:12.426 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:12.426 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:12.426 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:12.427 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:12.435 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:12.442+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:12.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:12.536 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:12.543+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:12.550 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:12.551 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:12.587 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:12.587 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:12.621 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:12.621 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:12.625 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:12.625 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:12.636 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:12.637 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:12.643+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:12.647 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:12.648 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:12.674 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:12.674 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:12.710 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:12.710 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:12.738 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:12.738 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:12.738 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:12.744+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:12.773 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:12.774 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:12.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:12.848 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-05-08T08:26:12.848 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:12.855+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:12.940 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:12.947+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:12.969 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:12.969 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:12.979 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:13.063 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:13.071+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:13.098 INFO:teuthology.orchestra.run.smithi031.stdout:5571 punch from 0xc33210f to 0xc33fd82, (0xdc73 bytes) 2024-05-08T08:26:13.102 INFO:teuthology.orchestra.run.smithi031.stdout:5574 read 0xb05385f thru 0xb05925d (0x59ff bytes) 2024-05-08T08:26:13.102 INFO:teuthology.orchestra.run.smithi031.stdout:5575 write 0x16dab7c thru 0x16ea4da (0xf95f bytes) 2024-05-08T08:26:13.128 INFO:teuthology.orchestra.run.smithi031.stdout:5576 read 0xcf1cfa5 thru 0xcf2798b (0xa9e7 bytes) 2024-05-08T08:26:13.128 INFO:teuthology.orchestra.run.smithi031.stdout:5577 write 0x9462afd thru 0x946773b (0x4c3f bytes) 2024-05-08T08:26:13.141 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:13.148+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:13.146 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:13.146 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:5580 punch from 0x9ef9725 to 0x9efd1f8, (0x3ad3 bytes) 2024-05-08T08:26:13.154 INFO:teuthology.orchestra.run.smithi031.stdout:5581 write 0x7ac2edc thru 0x7ac4986 (0x1aab bytes) 2024-05-08T08:26:13.157 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:13.162 INFO:teuthology.orchestra.run.smithi031.stdout:5582 write 0xc1b4354 thru 0xc1ba42c (0x60d9 bytes) 2024-05-08T08:26:13.180 INFO:teuthology.orchestra.run.smithi031.stdout:5583 trunc from 0xd8a5c12 to 0x86072d6 2024-05-08T08:26:13.213 INFO:teuthology.orchestra.run.smithi031.stdout:5584 punch from 0x5101cb5 to 0x5108430, (0x677b bytes) 2024-05-08T08:26:13.224 INFO:teuthology.orchestra.run.smithi031.stdout:5585 trunc from 0x86072d6 to 0x683359 2024-05-08T08:26:13.242 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:13.249+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:13.294 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:26:13.329 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:13.329 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:13.340 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:13.382 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:13.389+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:13.388 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:13.388 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:13.389 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:26:13.389 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:26:13.456 INFO:teuthology.orchestra.run.smithi031.stdout:5587 trunc from 0x683359 to 0x2944c0f 2024-05-08T08:26:13.456 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:13.463+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:13.468 INFO:teuthology.orchestra.run.smithi031.stdout:5591 trunc from 0x2944c0f to 0x8baa30d 2024-05-08T08:26:13.473 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:13.473 INFO:teuthology.orchestra.run.smithi031.stdout:5593 punch from 0x6d3bf66 to 0x6d47c02, (0xbc9c bytes) 2024-05-08T08:26:13.473 INFO:teuthology.orchestra.run.smithi031.stdout:5595 read 0x486521a thru 0x487414d (0xef34 bytes) 2024-05-08T08:26:13.473 INFO:teuthology.orchestra.run.smithi031.stdout:5596 read 0x56e5182 thru 0x56f3a6f (0xe8ee bytes) 2024-05-08T08:26:13.473 INFO:teuthology.orchestra.run.smithi031.stdout:5597 clone 34 order 23 su 131072 sc 8 2024-05-08T08:26:13.515 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:13.515 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:13.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:13.544 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:13.551+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:13.645 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:13.652+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:13.675 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:13.675 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:13.684 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:13.733 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 78, 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf", 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-05-08T08:24:44.143941+0000", 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-05-08T08:26:13.288474+0000", 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-05-08T08:25:44.118968+0000", 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-05-08T08:24:46.429925+0000", 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-05-08T08:26:13.734 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "jewel", 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-05-08T08:26:13.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:55.134472+0000", 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:26:13.736 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "16", 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:26:13.737 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:26:13.738 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:26:13.739 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-05-08T08:26:13.740 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 1.75, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 1.75, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:56.465515+0000", 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:26:13.741 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:26:13.742 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:26:13.743 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:26:13.744 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:26:13.745 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.9000000953674316, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.9000000953674316, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-05-08T08:26:13.746 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:25:03.054989+0000", 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:26:13.747 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:26:13.748 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "77", 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 35, 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 77, 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:26:13.749 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:26:13.750 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:26:13.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 1.75, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 1.75, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 1.1399999856948853, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 1, 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:26:13.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14afea42-356a-4163-ac3f-2e270e807917", 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 71, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.753 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.755 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/727789107", 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/727789107", 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/727789107", 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/727789107", 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "db77a923-8f49-4b47-b752-00b475917a0b", 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 13, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 30, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 54, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 69, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 32, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.758 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.760 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2489759853", 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2489759853", 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2489759853", 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2489759853", 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.762 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "3d508691-8e40-4592-82d0-eaf4c4c7d7dd", 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.582733154296875, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.763 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.765 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2736372115", 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2736372115", 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2736372115", 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2736372115", 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.767 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2545300a-428a-41ed-b19a-3a0cb9e5e755", 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 71, 2024-05-08T08:26:13.768 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.770 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.771 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/573482428", 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/573482428", 2024-05-08T08:26:13.772 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/573482428", 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/573482428", 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "886aeed5-4495-4064-8579-050d38a4ad1b", 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:13.773 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 71, 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6801", 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6803", 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.774 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6805", 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6807", 2024-05-08T08:26:13.775 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6813", 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6815", 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6809", 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6811", 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.777 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6803/3424463091", 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6807/3424463091", 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6815/3424463091", 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6811/3424463091", 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4d15cef0-9ee9-4069-a6c4-a650e873ee03", 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.778 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 18, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6816", 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.779 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6818", 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6820", 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6822", 2024-05-08T08:26:13.780 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6828", 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6830", 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6824", 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6826", 2024-05-08T08:26:13.782 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6818/3709461555", 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6822/3709461555", 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6830/3709461555", 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6826/3709461555", 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "910ac664-9573-4851-9e2b-89ac6015fdc1", 2024-05-08T08:26:13.783 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 32, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6800", 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6802", 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6804", 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6806", 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6812", 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6814", 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6808", 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.787 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6810", 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6802/405561966", 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6806/405561966", 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6814/405561966", 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6810/405561966", 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.788 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14cc2d94-48e4-47c9-896a-9be9281ef0c5", 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 71, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.789 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6817", 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6819", 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.790 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6821", 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6823", 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6829", 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6831", 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6825", 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:13.792 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6827", 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6819/570236217", 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6823/570236217", 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6831/570236217", 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6827/570236217", 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.793 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.572981+0000", 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.794 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.626543+0000", 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 32 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.638878+0000", 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:13.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.599293+0000", 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:26:13.796 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.565676+0000", 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.552124+0000", 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:13.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.531428+0000", 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:13.798 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.658391+0000", 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [], 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-05-08T08:26:13.799 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [], 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [], 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-05-08T08:26:13.800 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:26:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:26:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-05-08T08:26:13.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:13.801 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:13.802 INFO:tasks.thrashosds.thrasher:j is {'epoch': 78, 'fsid': '1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf', 'created': '2024-05-08T08:24:44.143941+0000', 'modified': '2024-05-08T08:26:13.288474+0000', 'last_up_change': '2024-05-08T08:25:44.118968+0000', 'last_in_change': '2024-05-08T08:24:46.429925+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:24:55.134472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:24:56.465515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:25:03.054989+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1.1399999856948853, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': '14afea42-356a-4163-ac3f-2e270e807917', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 727789107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 727789107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 727789107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 727789107}]}, 'public_addr': '172.21.15.31:6801/727789107', 'cluster_addr': '172.21.15.31:6803/727789107', 'heartbeat_back_addr': '172.21.15.31:6807/727789107', 'heartbeat_front_addr': '172.21.15.31:6805/727789107', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'db77a923-8f49-4b47-b752-00b475917a0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 30, 'up_from': 54, 'up_thru': 69, 'down_at': 32, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2489759853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2489759853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2489759853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2489759853}]}, 'public_addr': '172.21.15.31:6817/2489759853', 'cluster_addr': '172.21.15.31:6819/2489759853', 'heartbeat_back_addr': '172.21.15.31:6823/2489759853', 'heartbeat_front_addr': '172.21.15.31:6821/2489759853', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '3d508691-8e40-4592-82d0-eaf4c4c7d7dd', 'up': 1, 'in': 1, 'weight': 0.582733154296875, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2736372115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2736372115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2736372115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2736372115}]}, 'public_addr': '172.21.15.31:6809/2736372115', 'cluster_addr': '172.21.15.31:6811/2736372115', 'heartbeat_back_addr': '172.21.15.31:6815/2736372115', 'heartbeat_front_addr': '172.21.15.31:6813/2736372115', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2545300a-428a-41ed-b19a-3a0cb9e5e755', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 573482428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 573482428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 573482428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 573482428}]}, 'public_addr': '172.21.15.31:6825/573482428', 'cluster_addr': '172.21.15.31:6827/573482428', 'heartbeat_back_addr': '172.21.15.31:6831/573482428', 'heartbeat_front_addr': '172.21.15.31:6829/573482428', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '886aeed5-4495-4064-8579-050d38a4ad1b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6801', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6803', 'nonce': 3424463091}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6805', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6807', 'nonce': 3424463091}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6813', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6815', 'nonce': 3424463091}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6809', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6811', 'nonce': 3424463091}]}, 'public_addr': '172.21.15.149:6803/3424463091', 'cluster_addr': '172.21.15.149:6807/3424463091', 'heartbeat_back_addr': '172.21.15.149:6815/3424463091', 'heartbeat_front_addr': '172.21.15.149:6811/3424463091', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d15cef0-9ee9-4069-a6c4-a650e873ee03', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 18, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6816', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6818', 'nonce': 3709461555}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6820', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6822', 'nonce': 3709461555}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6828', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6830', 'nonce': 3709461555}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6824', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6826', 'nonce': 3709461555}]}, 'public_addr': '172.21.15.149:6818/3709461555', 'cluster_addr': '172.21.15.149:6822/3709461555', 'heartbeat_back_addr': '172.21.15.149:6830/3709461555', 'heartbeat_front_addr': '172.21.15.149:6826/3709461555', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '910ac664-9573-4851-9e2b-89ac6015fdc1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 32, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6800', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6802', 'nonce': 405561966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6804', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6806', 'nonce': 405561966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6812', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6814', 'nonce': 405561966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6808', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6810', 'nonce': 405561966}]}, 'public_addr': '172.21.15.149:6802/405561966', 'cluster_addr': '172.21.15.149:6806/405561966', 'heartbeat_back_addr': '172.21.15.149:6814/405561966', 'heartbeat_front_addr': '172.21.15.149:6810/405561966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '14cc2d94-48e4-47c9-896a-9be9281ef0c5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 71, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6817', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6819', 'nonce': 570236217}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6821', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6823', 'nonce': 570236217}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6829', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6831', 'nonce': 570236217}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6825', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6827', 'nonce': 570236217}]}, 'public_addr': '172.21.15.149:6819/570236217', 'cluster_addr': '172.21.15.149:6823/570236217', 'heartbeat_back_addr': '172.21.15.149:6831/570236217', 'heartbeat_front_addr': '172.21.15.149:6827/570236217', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.572981+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.626543+0000', 'dead_epoch': 32}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.638878+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.599293+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.565676+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.552124+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.531428+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.658391+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:26:13.803 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:26:13.803 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:13.809+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:13.834 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:13.834 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:13.843 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:13.902 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:13.910+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:13.951 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:13.951 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:13.960 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:14.004 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:14.011+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:14.067 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:14.067 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:14.077 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:14.118 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:14.125+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:14.130 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:26:14.131 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:26:14.142 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-05-08T08:26:13.299440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1153619858,"num_objects":1545,"num_object_clones":308,"num_object_copies":3090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1545,"num_whiteouts":505,"num_read":118859,"num_read_kb":741263,"num_write":18109,"num_write_kb":429181,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":19861750,"num_keys_recovered":34,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13945,"ondisk_log_size":13945,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":415128,"kb_used_data":62712,"kb_used_omap":24,"kb_used_meta":352295,"kb_avail":754559592,"statfs":{"total":773094113280,"available":772669022208,"internally_reserved":0,"allocated":64217088,"data_stored":592423425,"data_compressed":20594048,"data_compressed_allocated":47149056,"data_compressed_original":583249350,"omap_allocated":24704,"internal_metadata":360750976},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":153589472,"num_objects":132,"num_object_clones":4,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":2,"num_read":15537,"num_read_kb":17933,"num_write":860,"num_write_kb":9140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.030752"},"pg_stats":[{"pgid":"3.1f","version":"76'514","reported_seq":2921,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.290161+0000","last_change":"2024-05-08T08:25:06.628835+0000","last_active":"2024-05-08T08:26:10.290161+0000","last_peered":"2024-05-08T08:26:10.290161+0000","last_clean":"2024-05-08T08:26:10.290161+0000","last_became_active":"2024-05-08T08:25:04.610836+0000","last_became_peered":"2024-05-08T08:25:04.610836+0000","last_unstale":"2024-05-08T08:26:10.290161+0000","last_undegraded":"2024-05-08T08:26:10.290161+0000","last_fullsized":"2024-05-08T08:26:10.290161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:46.747770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011177400000000001,"stat_sum":{"num_bytes":24719885,"num_objects":43,"num_object_clones":6,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":6036,"num_read_kb":18288,"num_write":513,"num_write_kb":6912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"75'692","reported_seq":2673,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285905+0000","last_change":"2024-05-08T08:25:06.611278+0000","last_active":"2024-05-08T08:26:10.285905+0000","last_peered":"2024-05-08T08:26:10.285905+0000","last_clean":"2024-05-08T08:26:10.285905+0000","last_became_active":"2024-05-08T08:25:04.611994+0000","last_became_peered":"2024-05-08T08:25:04.611994+0000","last_unstale":"2024-05-08T08:26:10.285905+0000","last_undegraded":"2024-05-08T08:26:10.285905+0000","last_fullsized":"2024-05-08T08:26:10.285905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":692,"log_dups_size":0,"ondisk_log_size":692,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:39:23.334361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010168530000000001,"stat_sum":{"num_bytes":31061564,"num_objects":61,"num_object_clones":14,"num_object_copies":122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":61,"num_whiteouts":23,"num_read":4556,"num_read_kb":25769,"num_write":1130,"num_write_kb":17522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"75'467","reported_seq":1585,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285649+0000","last_change":"2024-05-08T08:26:04.255500+0000","last_active":"2024-05-08T08:26:10.285649+0000","last_peered":"2024-05-08T08:26:10.285649+0000","last_clean":"2024-05-08T08:26:10.285649+0000","last_became_active":"2024-05-08T08:26:04.254033+0000","last_became_peered":"2024-05-08T08:26:04.254033+0000","last_unstale":"2024-05-08T08:26:10.285649+0000","last_undegraded":"2024-05-08T08:26:10.285649+0000","last_fullsized":"2024-05-08T08:26:10.285649+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":467,"log_dups_size":0,"ondisk_log_size":467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:59:50.857646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001005171,"stat_sum":{"num_bytes":19377349,"num_objects":41,"num_object_clones":9,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":11,"num_read":2474,"num_read_kb":21236,"num_write":643,"num_write_kb":13722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"75'545","reported_seq":4581,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285379+0000","last_change":"2024-05-08T08:26:04.254805+0000","last_active":"2024-05-08T08:26:10.285379+0000","last_peered":"2024-05-08T08:26:10.285379+0000","last_clean":"2024-05-08T08:26:10.285379+0000","last_became_active":"2024-05-08T08:26:04.253888+0000","last_became_peered":"2024-05-08T08:26:04.253888+0000","last_unstale":"2024-05-08T08:26:10.285379+0000","last_undegraded":"2024-05-08T08:26:10.285379+0000","last_fullsized":"2024-05-08T08:26:10.285379+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:42:49.980925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010445210000000001,"stat_sum":{"num_bytes":100243149,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":13,"num_read":6142,"num_read_kb":61782,"num_write":759,"num_write_kb":22951,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"75'981","reported_seq":2042,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.290014+0000","last_change":"2024-05-08T08:25:06.628198+0000","last_active":"2024-05-08T08:26:10.290014+0000","last_peered":"2024-05-08T08:26:10.290014+0000","last_clean":"2024-05-08T08:26:10.290014+0000","last_became_active":"2024-05-08T08:25:04.612215+0000","last_became_peered":"2024-05-08T08:25:04.612215+0000","last_unstale":"2024-05-08T08:26:10.290014+0000","last_undegraded":"2024-05-08T08:26:10.290014+0000","last_fullsized":"2024-05-08T08:26:10.290014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:55.099598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064530100000000003,"stat_sum":{"num_bytes":25333232,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":5720,"num_read_kb":10788,"num_write":1597,"num_write_kb":8063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"75'350","reported_seq":10402,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285415+0000","last_change":"2024-05-08T08:25:48.158059+0000","last_active":"2024-05-08T08:26:10.285415+0000","last_peered":"2024-05-08T08:26:10.285415+0000","last_clean":"2024-05-08T08:26:10.285415+0000","last_became_active":"2024-05-08T08:25:48.157597+0000","last_became_peered":"2024-05-08T08:25:48.157597+0000","last_unstale":"2024-05-08T08:26:10.285415+0000","last_undegraded":"2024-05-08T08:26:10.285415+0000","last_fullsized":"2024-05-08T08:26:10.285415+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":350,"log_dups_size":0,"ondisk_log_size":350,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:34.630679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00080728699999999996,"stat_sum":{"num_bytes":55335067,"num_objects":50,"num_object_clones":8,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":16,"num_read":10168,"num_read_kb":27480,"num_write":417,"num_write_kb":15304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"77'325","reported_seq":1651,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-08T08:26:12.337764+0000","last_change":"2024-05-08T08:25:48.159653+0000","last_active":"2024-05-08T08:26:12.337764+0000","last_peered":"2024-05-08T08:26:12.337764+0000","last_clean":"2024-05-08T08:26:12.337764+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:26:12.337764+0000","last_undegraded":"2024-05-08T08:26:12.337764+0000","last_fullsized":"2024-05-08T08:26:12.337764+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":325,"log_dups_size":0,"ondisk_log_size":325,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:01:08.950382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040135600000000001,"stat_sum":{"num_bytes":22676438,"num_objects":45,"num_object_clones":9,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":13,"num_read":2119,"num_read_kb":21105,"num_write":379,"num_write_kb":9745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"75'298","reported_seq":6628,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.452193+0000","last_change":"2024-05-08T08:25:06.610175+0000","last_active":"2024-05-08T08:26:10.452193+0000","last_peered":"2024-05-08T08:26:10.452193+0000","last_clean":"2024-05-08T08:26:10.452193+0000","last_became_active":"2024-05-08T08:25:04.609185+0000","last_became_peered":"2024-05-08T08:25:04.609185+0000","last_unstale":"2024-05-08T08:26:10.452193+0000","last_undegraded":"2024-05-08T08:26:10.452193+0000","last_fullsized":"2024-05-08T08:26:10.452193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":298,"log_dups_size":0,"ondisk_log_size":298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:40:36.115928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000505687,"stat_sum":{"num_bytes":43955100,"num_objects":44,"num_object_clones":9,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":13,"num_read":6798,"num_read_kb":20639,"num_write":321,"num_write_kb":16193,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"76'650","reported_seq":1650,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.290207+0000","last_change":"2024-05-08T08:25:06.628387+0000","last_active":"2024-05-08T08:26:10.290207+0000","last_peered":"2024-05-08T08:26:10.290207+0000","last_clean":"2024-05-08T08:26:10.290207+0000","last_became_active":"2024-05-08T08:25:04.610818+0000","last_became_peered":"2024-05-08T08:25:04.610818+0000","last_unstale":"2024-05-08T08:26:10.290207+0000","last_undegraded":"2024-05-08T08:26:10.290207+0000","last_fullsized":"2024-05-08T08:26:10.290207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":650,"log_dups_size":0,"ondisk_log_size":650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:18.553191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069309899999999997,"stat_sum":{"num_bytes":33318481,"num_objects":47,"num_object_clones":5,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":3664,"num_read_kb":7013,"num_write":968,"num_write_kb":7944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"75'282","reported_seq":1396,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350925+0000","last_change":"2024-05-08T08:26:04.255114+0000","last_active":"2024-05-08T08:26:10.350925+0000","last_peered":"2024-05-08T08:26:10.350925+0000","last_clean":"2024-05-08T08:26:10.350925+0000","last_became_active":"2024-05-08T08:26:04.253934+0000","last_became_peered":"2024-05-08T08:26:04.253934+0000","last_unstale":"2024-05-08T08:26:10.350925+0000","last_undegraded":"2024-05-08T08:26:10.350925+0000","last_fullsized":"2024-05-08T08:26:10.350925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:24:49.353356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011464940000000001,"stat_sum":{"num_bytes":12686425,"num_objects":46,"num_object_clones":8,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":17,"num_read":1803,"num_read_kb":14143,"num_write":284,"num_write_kb":11695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.15","version":"75'293","reported_seq":3024,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350926+0000","last_change":"2024-05-08T08:25:06.796215+0000","last_active":"2024-05-08T08:26:10.350926+0000","last_peered":"2024-05-08T08:26:10.350926+0000","last_clean":"2024-05-08T08:26:10.350926+0000","last_became_active":"2024-05-08T08:25:04.606180+0000","last_became_peered":"2024-05-08T08:25:04.606180+0000","last_unstale":"2024-05-08T08:26:10.350926+0000","last_undegraded":"2024-05-08T08:26:10.350926+0000","last_fullsized":"2024-05-08T08:26:10.350926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":293,"log_dups_size":0,"ondisk_log_size":293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:39:37.688266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00027126400000000001,"stat_sum":{"num_bytes":50135547,"num_objects":46,"num_object_clones":10,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":15,"num_read":3725,"num_read_kb":17896,"num_write":331,"num_write_kb":5477,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"75'431","reported_seq":1148,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285639+0000","last_change":"2024-05-08T08:25:06.615415+0000","last_active":"2024-05-08T08:26:10.285639+0000","last_peered":"2024-05-08T08:26:10.285639+0000","last_clean":"2024-05-08T08:26:10.285639+0000","last_became_active":"2024-05-08T08:25:04.614760+0000","last_became_peered":"2024-05-08T08:25:04.614760+0000","last_unstale":"2024-05-08T08:26:10.285639+0000","last_undegraded":"2024-05-08T08:26:10.285639+0000","last_fullsized":"2024-05-08T08:26:10.285639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:54.385451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034964000000000002,"stat_sum":{"num_bytes":20252721,"num_objects":47,"num_object_clones":12,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":13,"num_read":1549,"num_read_kb":8783,"num_write":598,"num_write_kb":10405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":130,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T08:26:08.319519+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:26:08.319519+0000","last_peered":"2024-05-08T08:26:08.319519+0000","last_clean":"2024-05-08T08:26:08.319519+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:26:08.319519+0000","last_undegraded":"2024-05-08T08:26:08.319519+0000","last_fullsized":"2024-05-08T08:26:08.319519+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:32.338505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"75'318","reported_seq":7714,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285947+0000","last_change":"2024-05-08T08:26:04.251737+0000","last_active":"2024-05-08T08:26:10.285947+0000","last_peered":"2024-05-08T08:26:10.285947+0000","last_clean":"2024-05-08T08:26:10.285947+0000","last_became_active":"2024-05-08T08:26:04.250782+0000","last_became_peered":"2024-05-08T08:26:04.250782+0000","last_unstale":"2024-05-08T08:26:10.285947+0000","last_undegraded":"2024-05-08T08:26:10.285947+0000","last_fullsized":"2024-05-08T08:26:10.285947+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":318,"log_dups_size":0,"ondisk_log_size":318,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:27:46.909784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011708910000000001,"stat_sum":{"num_bytes":20146656,"num_objects":64,"num_object_clones":16,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":22,"num_read":7792,"num_read_kb":12443,"num_write":320,"num_write_kb":10063,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":130,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T08:26:08.319541+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:26:08.319541+0000","last_peered":"2024-05-08T08:26:08.319541+0000","last_clean":"2024-05-08T08:26:08.319541+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:26:08.319541+0000","last_undegraded":"2024-05-08T08:26:08.319541+0000","last_fullsized":"2024-05-08T08:26:08.319541+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:54.867771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"77'315","reported_seq":3996,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-08T08:26:13.129938+0000","last_change":"2024-05-08T08:26:01.242909+0000","last_active":"2024-05-08T08:26:13.129938+0000","last_peered":"2024-05-08T08:26:13.129938+0000","last_clean":"2024-05-08T08:26:13.129938+0000","last_became_active":"2024-05-08T08:26:01.241372+0000","last_became_peered":"2024-05-08T08:26:01.241372+0000","last_unstale":"2024-05-08T08:26:13.129938+0000","last_undegraded":"2024-05-08T08:26:13.129938+0000","last_fullsized":"2024-05-08T08:26:13.129938+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:29:12.966293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847699999999999,"stat_sum":{"num_bytes":40556494,"num_objects":45,"num_object_clones":11,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":17,"num_read":4353,"num_read_kb":29302,"num_write":339,"num_write_kb":6428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":135,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285732+0000","last_change":"2024-05-08T08:25:58.223928+0000","last_active":"2024-05-08T08:26:10.285732+0000","last_peered":"2024-05-08T08:26:10.285732+0000","last_clean":"2024-05-08T08:26:10.285732+0000","last_became_active":"2024-05-08T08:25:58.223356+0000","last_became_peered":"2024-05-08T08:25:58.223356+0000","last_unstale":"2024-05-08T08:26:10.285732+0000","last_undegraded":"2024-05-08T08:26:10.285732+0000","last_fullsized":"2024-05-08T08:26:10.285732+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:12:12.674757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"75'676","reported_seq":5628,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.286192+0000","last_change":"2024-05-08T08:25:06.611208+0000","last_active":"2024-05-08T08:26:10.286192+0000","last_peered":"2024-05-08T08:26:10.286192+0000","last_clean":"2024-05-08T08:26:10.286192+0000","last_became_active":"2024-05-08T08:25:04.608555+0000","last_became_peered":"2024-05-08T08:25:04.608555+0000","last_unstale":"2024-05-08T08:26:10.286192+0000","last_undegraded":"2024-05-08T08:26:10.286192+0000","last_fullsized":"2024-05-08T08:26:10.286192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":676,"log_dups_size":0,"ondisk_log_size":676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:37:07.191000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001195331,"stat_sum":{"num_bytes":51299769,"num_objects":62,"num_object_clones":11,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":17,"num_read":7950,"num_read_kb":65987,"num_write":947,"num_write_kb":38545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":131,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.286382+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:26:10.286382+0000","last_peered":"2024-05-08T08:26:10.286382+0000","last_clean":"2024-05-08T08:26:10.286382+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:26:10.286382+0000","last_undegraded":"2024-05-08T08:26:10.286382+0000","last_fullsized":"2024-05-08T08:26:10.286382+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"75'334","reported_seq":2099,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.289963+0000","last_change":"2024-05-08T08:25:48.159785+0000","last_active":"2024-05-08T08:26:10.289963+0000","last_peered":"2024-05-08T08:26:10.289963+0000","last_clean":"2024-05-08T08:26:10.289963+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:26:10.289963+0000","last_undegraded":"2024-05-08T08:26:10.289963+0000","last_fullsized":"2024-05-08T08:26:10.289963+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:15:38.063914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062277100000000005,"stat_sum":{"num_bytes":39764026,"num_objects":54,"num_object_clones":9,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":18,"num_read":2498,"num_read_kb":37993,"num_write":359,"num_write_kb":19539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"76'936","reported_seq":2096,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.422568+0000","last_change":"2024-05-08T08:25:06.628002+0000","last_active":"2024-05-08T08:26:10.422568+0000","last_peered":"2024-05-08T08:26:10.422568+0000","last_clean":"2024-05-08T08:26:10.422568+0000","last_became_active":"2024-05-08T08:25:04.615077+0000","last_became_peered":"2024-05-08T08:25:04.615077+0000","last_unstale":"2024-05-08T08:26:10.422568+0000","last_undegraded":"2024-05-08T08:26:10.422568+0000","last_fullsized":"2024-05-08T08:26:10.422568+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":936,"log_dups_size":0,"ondisk_log_size":936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:48:40.774827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057249000000000004,"stat_sum":{"num_bytes":13095920,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":23,"num_read":5325,"num_read_kb":13851,"num_write":1430,"num_write_kb":8219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":287,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350806+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:26:10.350806+0000","last_peered":"2024-05-08T08:26:10.350806+0000","last_clean":"2024-05-08T08:26:10.350806+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:26:10.350806+0000","last_undegraded":"2024-05-08T08:26:10.350806+0000","last_fullsized":"2024-05-08T08:26:10.350806+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-08T08:26:12.337692+0000","last_change":"2024-05-08T08:25:45.225724+0000","last_active":"2024-05-08T08:26:12.337692+0000","last_peered":"2024-05-08T08:26:12.337692+0000","last_clean":"2024-05-08T08:26:12.337692+0000","last_became_active":"2024-05-08T08:25:45.225388+0000","last_became_peered":"2024-05-08T08:25:45.225388+0000","last_unstale":"2024-05-08T08:26:12.337692+0000","last_undegraded":"2024-05-08T08:26:12.337692+0000","last_fullsized":"2024-05-08T08:26:12.337692+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:54.551477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"76'300","reported_seq":1320,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-05-08T08:26:12.337701+0000","last_change":"2024-05-08T08:25:48.160149+0000","last_active":"2024-05-08T08:26:12.337701+0000","last_peered":"2024-05-08T08:26:12.337701+0000","last_clean":"2024-05-08T08:26:12.337701+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:26:12.337701+0000","last_undegraded":"2024-05-08T08:26:12.337701+0000","last_fullsized":"2024-05-08T08:26:12.337701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":300,"log_dups_size":0,"ondisk_log_size":300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:57:56.745928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034192400000000001,"stat_sum":{"num_bytes":28830705,"num_objects":47,"num_object_clones":11,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":16,"num_read":869,"num_read_kb":14269,"num_write":342,"num_write_kb":11485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350865+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:26:10.350865+0000","last_peered":"2024-05-08T08:26:10.350865+0000","last_clean":"2024-05-08T08:26:10.350865+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:26:10.350865+0000","last_undegraded":"2024-05-08T08:26:10.350865+0000","last_fullsized":"2024-05-08T08:26:10.350865+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"75'516","reported_seq":4207,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285523+0000","last_change":"2024-05-08T08:25:06.615595+0000","last_active":"2024-05-08T08:26:10.285523+0000","last_peered":"2024-05-08T08:26:10.285523+0000","last_clean":"2024-05-08T08:26:10.285523+0000","last_became_active":"2024-05-08T08:25:04.611900+0000","last_became_peered":"2024-05-08T08:25:04.611900+0000","last_unstale":"2024-05-08T08:26:10.285523+0000","last_undegraded":"2024-05-08T08:26:10.285523+0000","last_fullsized":"2024-05-08T08:26:10.285523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":516,"log_dups_size":0,"ondisk_log_size":516,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:21:58.646342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050119499999999996,"stat_sum":{"num_bytes":26801503,"num_objects":43,"num_object_clones":9,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":14,"num_read":5161,"num_read_kb":26028,"num_write":613,"num_write_kb":16828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":19,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.289869+0000","last_change":"2024-05-08T08:26:04.255319+0000","last_active":"2024-05-08T08:26:10.289869+0000","last_peered":"2024-05-08T08:26:10.289869+0000","last_clean":"2024-05-08T08:26:10.289869+0000","last_became_active":"2024-05-08T08:26:04.255074+0000","last_became_peered":"2024-05-08T08:26:04.255074+0000","last_unstale":"2024-05-08T08:26:10.289869+0000","last_undegraded":"2024-05-08T08:26:10.289869+0000","last_fullsized":"2024-05-08T08:26:10.289869+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:52:16.514474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"76'254","reported_seq":1284,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.286153+0000","last_change":"2024-05-08T08:25:06.610137+0000","last_active":"2024-05-08T08:26:10.286153+0000","last_peered":"2024-05-08T08:26:10.286153+0000","last_clean":"2024-05-08T08:26:10.286153+0000","last_became_active":"2024-05-08T08:25:04.606801+0000","last_became_peered":"2024-05-08T08:25:04.606801+0000","last_unstale":"2024-05-08T08:26:10.286153+0000","last_undegraded":"2024-05-08T08:26:10.286153+0000","last_fullsized":"2024-05-08T08:26:10.286153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:16:58.012618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042592699999999999,"stat_sum":{"num_bytes":65596969,"num_objects":50,"num_object_clones":12,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":21,"num_read":953,"num_read_kb":47650,"num_write":254,"num_write_kb":21020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":132,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350808+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:26:10.350808+0000","last_peered":"2024-05-08T08:26:10.350808+0000","last_clean":"2024-05-08T08:26:10.350808+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:26:10.350808+0000","last_undegraded":"2024-05-08T08:26:10.350808+0000","last_fullsized":"2024-05-08T08:26:10.350808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"75'326","reported_seq":2904,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285996+0000","last_change":"2024-05-08T08:25:48.159093+0000","last_active":"2024-05-08T08:26:10.285996+0000","last_peered":"2024-05-08T08:26:10.285996+0000","last_clean":"2024-05-08T08:26:10.285996+0000","last_became_active":"2024-05-08T08:25:48.158198+0000","last_became_peered":"2024-05-08T08:25:48.158198+0000","last_unstale":"2024-05-08T08:26:10.285996+0000","last_undegraded":"2024-05-08T08:26:10.285996+0000","last_fullsized":"2024-05-08T08:26:10.285996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:07:05.524877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001097338,"stat_sum":{"num_bytes":35268197,"num_objects":59,"num_object_clones":11,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":19,"num_read":3999,"num_read_kb":25147,"num_write":358,"num_write_kb":20344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"75'277","reported_seq":1244,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.285374+0000","last_change":"2024-05-08T08:25:06.615379+0000","last_active":"2024-05-08T08:26:10.285374+0000","last_peered":"2024-05-08T08:26:10.285374+0000","last_clean":"2024-05-08T08:26:10.285374+0000","last_became_active":"2024-05-08T08:25:04.901435+0000","last_became_peered":"2024-05-08T08:25:04.901435+0000","last_unstale":"2024-05-08T08:26:10.285374+0000","last_undegraded":"2024-05-08T08:26:10.285374+0000","last_fullsized":"2024-05-08T08:26:10.285374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":277,"log_dups_size":0,"ondisk_log_size":277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:03:36.168182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039550799999999998,"stat_sum":{"num_bytes":14289174,"num_objects":42,"num_object_clones":9,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42,"num_whiteouts":16,"num_read":2174,"num_read_kb":7284,"num_write":272,"num_write_kb":4098,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"75'462","reported_seq":1207,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T08:26:08.764058+0000","last_change":"2024-05-08T08:25:06.612336+0000","last_active":"2024-05-08T08:26:08.764058+0000","last_peered":"2024-05-08T08:26:08.764058+0000","last_clean":"2024-05-08T08:26:08.764058+0000","last_became_active":"2024-05-08T08:25:04.615227+0000","last_became_peered":"2024-05-08T08:25:04.615227+0000","last_unstale":"2024-05-08T08:26:08.764058+0000","last_undegraded":"2024-05-08T08:26:08.764058+0000","last_fullsized":"2024-05-08T08:26:08.764058+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:30:31.300592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034968800000000002,"stat_sum":{"num_bytes":30709416,"num_objects":50,"num_object_clones":9,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":14,"num_read":1754,"num_read_kb":18511,"num_write":610,"num_write_kb":9735,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"76'268","reported_seq":1640,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.286142+0000","last_change":"2024-05-08T08:25:06.610868+0000","last_active":"2024-05-08T08:26:10.286142+0000","last_peered":"2024-05-08T08:26:10.286142+0000","last_clean":"2024-05-08T08:26:10.286142+0000","last_became_active":"2024-05-08T08:25:04.608456+0000","last_became_peered":"2024-05-08T08:25:04.608456+0000","last_unstale":"2024-05-08T08:26:10.286142+0000","last_undegraded":"2024-05-08T08:26:10.286142+0000","last_fullsized":"2024-05-08T08:26:10.286142+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":268,"log_dups_size":0,"ondisk_log_size":268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:12:18.075696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001112149,"stat_sum":{"num_bytes":40928779,"num_objects":58,"num_object_clones":12,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":21,"num_read":1298,"num_read_kb":15823,"num_write":269,"num_write_kb":10634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"76'264","reported_seq":3310,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.286353+0000","last_change":"2024-05-08T08:26:04.251506+0000","last_active":"2024-05-08T08:26:10.286353+0000","last_peered":"2024-05-08T08:26:10.286353+0000","last_clean":"2024-05-08T08:26:10.286353+0000","last_became_active":"2024-05-08T08:26:04.250525+0000","last_became_peered":"2024-05-08T08:26:04.250525+0000","last_unstale":"2024-05-08T08:26:10.286353+0000","last_undegraded":"2024-05-08T08:26:10.286353+0000","last_fullsized":"2024-05-08T08:26:10.286353+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":264,"log_dups_size":0,"ondisk_log_size":264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:11:25.262583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046376499999999999,"stat_sum":{"num_bytes":71278641,"num_objects":35,"num_object_clones":5,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":3032,"num_read_kb":38879,"num_write":333,"num_write_kb":23663,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.d","version":"75'265","reported_seq":895,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350997+0000","last_change":"2024-05-08T08:25:06.796419+0000","last_active":"2024-05-08T08:26:10.350997+0000","last_peered":"2024-05-08T08:26:10.350997+0000","last_clean":"2024-05-08T08:26:10.350997+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:26:10.350997+0000","last_undegraded":"2024-05-08T08:26:10.350997+0000","last_fullsized":"2024-05-08T08:26:10.350997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:13:35.442681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046517299999999999,"stat_sum":{"num_bytes":18041535,"num_objects":46,"num_object_clones":11,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":535,"num_read_kb":11318,"num_write":267,"num_write_kb":5773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"76'445","reported_seq":3179,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.351021+0000","last_change":"2024-05-08T08:25:06.796635+0000","last_active":"2024-05-08T08:26:10.351021+0000","last_peered":"2024-05-08T08:26:10.351021+0000","last_clean":"2024-05-08T08:26:10.351021+0000","last_became_active":"2024-05-08T08:25:04.615004+0000","last_became_peered":"2024-05-08T08:25:04.615004+0000","last_unstale":"2024-05-08T08:26:10.351021+0000","last_undegraded":"2024-05-08T08:26:10.351021+0000","last_fullsized":"2024-05-08T08:26:10.351021+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":445,"log_dups_size":0,"ondisk_log_size":445,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:46:45.913599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060116800000000001,"stat_sum":{"num_bytes":73328866,"num_objects":46,"num_object_clones":5,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":14,"num_read":4115,"num_read_kb":45169,"num_write":482,"num_write_kb":26742,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"75'273","reported_seq":1288,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.351040+0000","last_change":"2024-05-08T08:25:06.796737+0000","last_active":"2024-05-08T08:26:10.351040+0000","last_peered":"2024-05-08T08:26:10.351040+0000","last_clean":"2024-05-08T08:26:10.351040+0000","last_became_active":"2024-05-08T08:25:04.615942+0000","last_became_peered":"2024-05-08T08:25:04.615942+0000","last_unstale":"2024-05-08T08:26:10.351040+0000","last_undegraded":"2024-05-08T08:26:10.351040+0000","last_fullsized":"2024-05-08T08:26:10.351040+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":273,"log_dups_size":0,"ondisk_log_size":273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:38.801174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065917600000000003,"stat_sum":{"num_bytes":31160919,"num_objects":37,"num_object_clones":6,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":10,"num_read":1271,"num_read_kb":4813,"num_write":314,"num_write_kb":2985,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"75'603","reported_seq":2271,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-05-08T08:26:09.639748+0000","last_change":"2024-05-08T08:25:06.612297+0000","last_active":"2024-05-08T08:26:09.639748+0000","last_peered":"2024-05-08T08:26:09.639748+0000","last_clean":"2024-05-08T08:26:09.639748+0000","last_became_active":"2024-05-08T08:25:04.616212+0000","last_became_peered":"2024-05-08T08:25:04.616212+0000","last_unstale":"2024-05-08T08:26:09.639748+0000","last_undegraded":"2024-05-08T08:26:09.639748+0000","last_fullsized":"2024-05-08T08:26:09.639748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:59:02.260145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031704699999999999,"stat_sum":{"num_bytes":11655185,"num_objects":43,"num_object_clones":8,"num_object_copies":86,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43,"num_whiteouts":10,"num_read":3600,"num_read_kb":12423,"num_write":907,"num_write_kb":9089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"75'282","reported_seq":1105,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350819+0000","last_change":"2024-05-08T08:25:06.796231+0000","last_active":"2024-05-08T08:26:10.350819+0000","last_peered":"2024-05-08T08:26:10.350819+0000","last_clean":"2024-05-08T08:26:10.350819+0000","last_became_active":"2024-05-08T08:25:04.615031+0000","last_became_peered":"2024-05-08T08:25:04.615031+0000","last_unstale":"2024-05-08T08:26:10.350819+0000","last_undegraded":"2024-05-08T08:26:10.350819+0000","last_fullsized":"2024-05-08T08:26:10.350819+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:58:08.513652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040064000000000001,"stat_sum":{"num_bytes":27639370,"num_objects":48,"num_object_clones":10,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":15,"num_read":902,"num_read_kb":19900,"num_write":326,"num_write_kb":8420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"72'285","reported_seq":1345,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.289827+0000","last_change":"2024-05-08T08:25:06.628036+0000","last_active":"2024-05-08T08:26:10.289827+0000","last_peered":"2024-05-08T08:26:10.289827+0000","last_clean":"2024-05-08T08:26:10.289827+0000","last_became_active":"2024-05-08T08:25:04.609651+0000","last_became_peered":"2024-05-08T08:25:04.609651+0000","last_unstale":"2024-05-08T08:26:10.289827+0000","last_undegraded":"2024-05-08T08:26:10.289827+0000","last_fullsized":"2024-05-08T08:26:10.289827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:54:41.981650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050895200000000002,"stat_sum":{"num_bytes":22557866,"num_objects":44,"num_object_clones":10,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":14,"num_read":1477,"num_read_kb":21727,"num_write":350,"num_write_kb":13886,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"76'601","reported_seq":3652,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-05-08T08:26:10.350965+0000","last_change":"2024-05-08T08:25:06.796497+0000","last_active":"2024-05-08T08:26:10.350965+0000","last_peered":"2024-05-08T08:26:10.350965+0000","last_clean":"2024-05-08T08:26:10.350965+0000","last_became_active":"2024-05-08T08:25:04.616040+0000","last_became_peered":"2024-05-08T08:25:04.616040+0000","last_unstale":"2024-05-08T08:26:10.350965+0000","last_undegraded":"2024-05-08T08:26:10.350965+0000","last_fullsized":"2024-05-08T08:26:10.350965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":601,"log_dups_size":0,"ondisk_log_size":601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:23:23.857504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050921500000000004,"stat_sum":{"num_bytes":50984542,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":16,"num_read":4931,"num_read_kb":17724,"num_write":918,"num_write_kb":13336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1153029490,"num_objects":1543,"num_object_clones":308,"num_object_copies":3086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1543,"num_whiteouts":505,"num_read":118743,"num_read_kb":741163,"num_write":17950,"num_write_kb":426765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":19861750,"num_keys_recovered":34,"num_objects_omap":35,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57425920,"data_stored":589245609,"data_compressed":20596544,"data_compressed_allocated":47108096,"data_compressed_original":583130566,"omap_allocated":9107,"internal_metadata":0},"log_size":13828,"ondisk_log_size":13828,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574866,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36540,"kb_used_data":9524,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94335300,"statfs":{"total":96636764160,"available":96599347200,"internally_reserved":0,"allocated":9752576,"data_stored":95445693,"data_compressed":3515302,"data_compressed_allocated":7290880,"data_compressed_original":94070316,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38464,"kb_used_data":3116,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333376,"statfs":{"total":96636764160,"available":96597377024,"internally_reserved":0,"allocated":3190784,"data_stored":23504419,"data_compressed":786570,"data_compressed_allocated":1830912,"data_compressed_original":22844416,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53864,"kb_used_data":10204,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94317976,"statfs":{"total":96636764160,"available":96581607424,"internally_reserved":0,"allocated":10448896,"data_stored":97353693,"data_compressed":3245170,"data_compressed_allocated":7962624,"data_compressed_original":95972402,"omap_allocated":1594,"internal_metadata":44693958},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81872,"kb_used_data":10440,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94289968,"statfs":{"total":96636764160,"available":96552927232,"internally_reserved":0,"allocated":10690560,"data_stored":97564538,"data_compressed":3696018,"data_compressed_allocated":7884800,"data_compressed_original":96017904,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55544,"kb_used_data":11884,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":94316296,"statfs":{"total":96636764160,"available":96579887104,"internally_reserved":0,"allocated":12169216,"data_stored":112028846,"data_compressed":3913755,"data_compressed_allocated":9502720,"data_compressed_original":110575770,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2117632,"data_stored":13082589,"data_compressed":483768,"data_compressed_allocated":950272,"data_compressed_original":12511528,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928233991,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39852,"kb_used_data":7008,"kb_used_omap":13,"kb_used_meta":32818,"kb_avail":94331988,"statfs":{"total":96636764160,"available":96595955712,"internally_reserved":0,"allocated":7176192,"data_stored":69933871,"data_compressed":2034965,"data_compressed_allocated":5378048,"data_compressed_original":68988551,"omap_allocated":13550,"internal_metadata":33606418},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574866,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":75740,"kb_used_data":8468,"kb_used_omap":1,"kb_used_meta":67262,"kb_avail":94296100,"statfs":{"total":96636764160,"available":96559206400,"internally_reserved":0,"allocated":8671232,"data_stored":83509776,"data_compressed":2918500,"data_compressed_allocated":6348800,"data_compressed_original":82268463,"omap_allocated":1595,"internal_metadata":68876741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":7811072,"data_stored":83120646,"data_compressed":2918500,"data_compressed_allocated":6348800,"data_compressed_original":82268463,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6270976,"data_stored":69969401,"data_compressed":2042404,"data_compressed_allocated":5373952,"data_compressed_original":69443207,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1245184,"data_stored":12682038,"data_compressed":483768,"data_compressed_allocated":950272,"data_compressed_original":12511528,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11313152,"data_stored":111639778,"data_compressed":3913755,"data_compressed_allocated":9502720,"data_compressed_original":110575770,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":9904128,"data_stored":97217270,"data_compressed":3696018,"data_compressed_allocated":7884800,"data_compressed_original":96017904,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9637888,"data_stored":96995156,"data_compressed":3245170,"data_compressed_allocated":7962624,"data_compressed_original":95972402,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2334720,"data_stored":23114108,"data_compressed":786570,"data_compressed_allocated":1830912,"data_compressed_original":22844416,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8908800,"data_stored":94507212,"data_compressed":3510359,"data_compressed_allocated":7254016,"data_compressed_original":93496876,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:26:14.144 INFO:tasks.thrashosds.thrasher:Setting 3.a to [3, 4] 2024-05-08T08:26:14.144 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.a', '3', '4'] 2024-05-08T08:26:14.145 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.a 3 4 2024-05-08T08:26:14.205 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:14.212+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:14.224 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:14.224 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:14.233 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:14.306 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:14.313+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:14.368 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone33 from 0x8baa30d (overlap 0x683359) to 0x535c47 2024-05-08T08:26:14.389 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:14.389 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:14.399 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:14.437 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-05-08T08:26:14.443 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:14.450+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:14.445 INFO:teuthology.orchestra.run.smithi031.stdout:5598 punch from 0x8ba2907 to 0x8ba765c, (0x4d55 bytes) 2024-05-08T08:26:14.460 INFO:teuthology.orchestra.run.smithi031.stdout:5599 writesame 0xc0c86ce thru 0xc0cd0f1 data_size 0xd(0x4a24 bytes) 2024-05-08T08:26:14.460 INFO:teuthology.orchestra.run.smithi031.stdout:5600 write 0x6910f36 thru 0x691ce05 (0xbed0 bytes) 2024-05-08T08:26:14.466 INFO:teuthology.orchestra.run.smithi031.stdout:5602 read 0x253c1bb thru 0x254b870 (0xf6b6 bytes) 2024-05-08T08:26:14.466 INFO:teuthology.orchestra.run.smithi031.stdout:5603 punch from 0x5119b8a to 0x5126c6c, (0xd0e2 bytes) 2024-05-08T08:26:14.466 INFO:teuthology.orchestra.run.smithi031.stdout:5604 read 0x19177cd thru 0x1920d1d (0x9551 bytes) 2024-05-08T08:26:14.471 INFO:teuthology.orchestra.run.smithi031.stdout:5605 write 0xc4077a4 thru 0xc410c00 (0x945d bytes) 2024-05-08T08:26:14.474 INFO:teuthology.orchestra.run.smithi031.stdout:5606 writesame 0xb2ba715 thru 0xb2c0e22 data_size 0x3(0x670e bytes) 2024-05-08T08:26:14.481 INFO:teuthology.orchestra.run.smithi031.stdout:5607 writesame 0x7a8f981 thru 0x7a9eb38 data_size 0xb6(0xf1b8 bytes) 2024-05-08T08:26:14.486 INFO:teuthology.orchestra.run.smithi031.stdout:5610 writesame 0x851edb thru 0x85ce18 data_size 0x3(0xaf3e bytes) 2024-05-08T08:26:14.505 INFO:teuthology.orchestra.run.smithi031.stdout:5612 write 0x23352da thru 0x23391b9 (0x3ee0 bytes) 2024-05-08T08:26:14.508 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:14.515+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:14.516 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:14.516 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:14.518 INFO:teuthology.orchestra.run.smithi031.stdout:5613 trunc from 0xc410c01 to 0xc1df3ba 2024-05-08T08:26:14.524 INFO:teuthology.orchestra.run.smithi031.stdout:5615 read 0x3fbfc90 thru 0x3fc8f13 (0x9284 bytes) 2024-05-08T08:26:14.524 INFO:teuthology.orchestra.run.smithi031.stdout:5616 write 0x654e288 thru 0x6556df0 (0x8b69 bytes) 2024-05-08T08:26:14.526 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:14.541 INFO:teuthology.orchestra.run.smithi031.stdout:5618 trunc from 0xc1df3ba to 0x1c24f1a 2024-05-08T08:26:14.609 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:14.617+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:14.619 INFO:teuthology.orchestra.run.smithi031.stdout:5622 writesame 0x87b6817 thru 0x87bbcdc data_size 0x3(0x54c6 bytes) 2024-05-08T08:26:14.619 INFO:teuthology.orchestra.run.smithi031.stdout:5623 read 0x30636ac thru 0x306a842 (0x7197 bytes) 2024-05-08T08:26:14.619 INFO:teuthology.orchestra.run.smithi031.stdout:5624 punch from 0x14f04f to 0x15709e, (0x804f bytes) 2024-05-08T08:26:14.634 INFO:teuthology.orchestra.run.smithi031.stdout:5625 writesame 0xd674dd4 thru 0xd678d3d data_size 0x1(0x3f6a bytes) 2024-05-08T08:26:14.634 INFO:teuthology.orchestra.run.smithi031.stdout:5626 read 0x9272f4c thru 0x927536a (0x241f bytes) 2024-05-08T08:26:14.634 INFO:teuthology.orchestra.run.smithi031.stdout:5627 writesame 0xa48f451 thru 0xa494ebe data_size 0x19(0x5a6e bytes) 2024-05-08T08:26:14.635 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:14.635 INFO:teuthology.orchestra.run.smithi031.stdout:5631 punch from 0x8893079 to 0x88a0d88, (0xdd0f bytes) 2024-05-08T08:26:14.635 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:14.635 INFO:teuthology.orchestra.run.smithi031.stdout:5635 write 0xa9291ba thru 0xa92d727 (0x456e bytes) 2024-05-08T08:26:14.639 INFO:teuthology.orchestra.run.smithi031.stdout:5636 punch from 0x9c20631 to 0x9c22b72, (0x2541 bytes) 2024-05-08T08:26:14.639 INFO:teuthology.orchestra.run.smithi031.stdout:5639 punch from 0x7e8e9cd to 0x7e9caaf, (0xe0e2 bytes) 2024-05-08T08:26:14.639 INFO:teuthology.orchestra.run.smithi031.stdout:5641 punch from 0x74a9022 to 0x74b8095, (0xf073 bytes) 2024-05-08T08:26:14.639 INFO:teuthology.orchestra.run.smithi031.stdout:5644 punch from 0x10ef6bc to 0x10f9384, (0x9cc8 bytes) 2024-05-08T08:26:14.639 INFO:teuthology.orchestra.run.smithi031.stdout:5647 read 0xcb117c0 thru 0xcb1d133 (0xb974 bytes) 2024-05-08T08:26:14.639 INFO:teuthology.orchestra.run.smithi031.stdout:5648 clone 35 order 20 su 16384 sc 2 2024-05-08T08:26:14.677 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:14.678 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:14.686 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:14.730 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:14.737+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:14.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:14.791 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:14.792 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:14.801 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:14.841 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:14.848+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:14.912 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:14.920+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:14.916 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:14.916 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:14.934 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:14.934 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:14.945 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:14.945 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:14.951 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:14.951 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:14.954 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:14.968 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:14.969 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:14.986 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:14.987 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:15.004 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:15.004 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:15.013 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:15.020+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:15.022 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:15.022 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:15.041 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:15.042 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290285037, 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:15.064 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.066 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:15.068 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290264319, 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.070 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:15.070 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.071 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:15.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.071 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.071 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.071 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:15.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:15.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:15.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:15.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:15.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.075 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290236382, 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.076 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:15.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:15.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290214619, 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.081 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290192040999999, 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.085 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:15.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.089 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290151972, 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.090 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:15.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:15.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290129664, 2024-05-08T08:26:15.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:15.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:15.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290101083, 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:15.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:15.103 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.29007683, 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:15.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.105 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.290044432, 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.108 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:15.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:15.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:15.112 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.288706627, 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:15.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.114 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:15.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.288684609000001, 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.288642250000001, 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.121 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.122 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:15.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.288600731000001, 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.126 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.130 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.286622635000001, 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:15.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.286611341, 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.135 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:15.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:15.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.282523564, 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:15.143 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.282508087, 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.144 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.277389312, 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.148 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:15.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:15.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.277111382999999, 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.152 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:15.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.156 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:15.158 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:15.158 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:15.165+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:15.258 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:15.265+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:15.264 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:15.264 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:15.264 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:15.264 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:15.273 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:15.306 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.a pg_upmap mapping to [3,4] 2024-05-08T08:26:15.320 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:26:15.359 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:15.366+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:15.421 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:15.421 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:15.421 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:15.421 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:15.422 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:15.430 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:15.460 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:15.468+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:15.562 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:15.569+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:15.578 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:15.578 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:15.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 78.018966595999999, 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.579 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:15.585 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794861289, 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.586 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794776134999999, 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.590 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.591 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.592 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794748229, 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.595 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.596 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:15.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.599 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794719828, 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.600 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.603 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794635689, 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:15.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:15.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:15.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:15.607 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794555194000001, 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.608 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:15.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:15.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794522614, 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.612 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:15.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:15.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.615 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794469423000001, 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.616 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794443444000001, 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.620 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:15.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:15.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:15.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794388112, 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.624 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:15.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:15.628 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794238078999999, 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:15.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794188419999999, 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.633 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794153851000001, 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.637 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:15.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:15.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794131844000001, 2024-05-08T08:26:15.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:15.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.645 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794108870000001, 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.647 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.794079192, 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:15.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:15.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.653 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.653 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:15.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:15.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:15.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.792856614, 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.656 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:15.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.79284612, 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.660 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:15.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:15.663 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 13.787727284000001, 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:15.664 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:15.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.668 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:15.669 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:15.669 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:15.670 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:15.671 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:15.677+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:15.771 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:15.778+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:15.780 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:15.780 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:15.780 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:15.789 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:15.872 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:15.880+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:15.938 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:15.938 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:15.938 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:15.947 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:15.973 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:15.980+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:16.074 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:16.081+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:16.100 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50476 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 161792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191184445000001, 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018840722000000001, 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50476 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915327+0000", 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:16.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915329+0000", 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915336+0000", 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4880000000000001e-06 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915342+0000", 2024-05-08T08:26:16.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.933988+0000", 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018646593 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934072+0000", 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3626000000000002e-05 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934167+0000", 2024-05-08T08:26:16.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5087999999999995e-05 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50477 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 171008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191161973, 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019055283999999999, 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50477 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:16.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915351+0000", 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.311e-06 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915363+0000", 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2031000000000001e-05 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915369+0000", 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6930000000000004e-06 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934223+0000", 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018853741 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934307+0000", 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4154999999999996e-05 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934404+0000", 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6973999999999997e-05 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.109 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50478 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 180224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191133353000001, 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019257972000000002, 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50478 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915378+0000", 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915387+0000", 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999993e-06 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915396+0000", 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2499999999999995e-06 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915401+0000", 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.259e-06 2024-05-08T08:26:16.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934434+0000", 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019032970999999999 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934535+0000", 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100479 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934635+0000", 2024-05-08T08:26:16.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100445 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50479 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 189440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191101094, 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019428513000000001, 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.114 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50479 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915413+0000", 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915419+0000", 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7320000000000001e-06 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915424+0000", 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934664+0000", 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019239903999999999 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934740+0000", 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5833000000000004e-05 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934838+0000", 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7832999999999997e-05 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50480 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 198656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.118 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191079182999999, 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019651075, 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50480 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915432+0000", 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915434+0000", 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.221e-06 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915440+0000", 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4770000000000002e-06 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915444+0000", 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8330000000000001e-06 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934880+0000", 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019435555 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934984+0000", 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010403099999999999 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935083+0000", 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8566000000000005e-05 2024-05-08T08:26:16.122 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50481 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191059245000002, 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019849193000000001, 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.123 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50481 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915452+0000", 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915454+0000", 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3719999999999999e-06 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915461+0000", 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6290000000000004e-06 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915466+0000", 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935126+0000", 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019660074 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935221+0000", 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4598000000000005e-05 2024-05-08T08:26:16.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935301+0000", 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.996e-05 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50482 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191037666, 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020069704000000001, 2024-05-08T08:26:16.127 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50482 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915476+0000", 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-05-08T08:26:16.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915482+0000", 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4030000000000004e-06 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915487+0000", 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9219999999999996e-06 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935343+0000", 2024-05-08T08:26:16.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019856492999999999 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935433+0000", 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9485000000000001e-05 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935543+0000", 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109832 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50483 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 226304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.191017340999998, 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020296027000000001, 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50483 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.132 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915494+0000", 2024-05-08T08:26:16.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915496+0000", 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915502+0000", 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7939999999999999e-06 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915507+0000", 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935582+0000", 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020075790999999999 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935672+0000", 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0074999999999994e-05 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935789+0000", 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011681299999999999 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50484 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.190997482, 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020571613999999998, 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.136 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50484 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915514+0000", 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915516+0000", 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2809999999999998e-06 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915649+0000", 2024-05-08T08:26:16.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000132776 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915668+0000", 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9253e-05 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935846+0000", 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020177820999999999 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935950+0000", 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104478 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936085+0000", 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000134618 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50485 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.190830796, 2024-05-08T08:26:16.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020672174000000001, 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50485 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4700000000000001e-07 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915684+0000", 2024-05-08T08:26:16.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5820000000000002e-06 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915692+0000", 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1829999999999994e-06 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915706+0000", 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3747999999999999e-05 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936120+0000", 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020414472999999999 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936205+0000", 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4561000000000006e-05 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936352+0000", 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014698000000000001 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.144 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50486 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.190794918000002, 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020885167, 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50486 2024-05-08T08:26:16.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915724+0000", 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4200000000000001e-06 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915734+0000", 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0796e-05 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915748+0000", 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3318e-05 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936388+0000", 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020640642000000001 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936491+0000", 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010300400000000001 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936601+0000", 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109523 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50737 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 208896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.182842010000002, 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018673419, 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.149 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50737 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923672+0000", 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.252e-06 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.151 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923679+0000", 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923737+0000", 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8035000000000003e-05 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942158+0000", 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018421334000000001 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942240+0000", 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1409000000000006e-05 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942342+0000", 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102252 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50738 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:16.153 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.182758662000001, 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018832728, 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50738 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:16.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923753+0000", 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-07 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923756+0000", 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4510000000000001e-06 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923766+0000", 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.049e-05 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923773+0000", 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942387+0000", 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018614117999999999 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942475+0000", 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7874999999999995e-05 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942585+0000", 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011014300000000001 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50739 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.182693698000001, 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990561999999999, 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50739 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923828+0000", 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0988999999999999e-05 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923837+0000", 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6889999999999993e-06 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923851+0000", 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3528e-05 2024-05-08T08:26:16.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942618+0000", 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018766998999999999 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942704+0000", 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5995000000000006e-05 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942807+0000", 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103837 2024-05-08T08:26:16.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50740 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.182650314, 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019168166, 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.162 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50740 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923861+0000", 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923863+0000", 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923871+0000", 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0569999999999993e-06 2024-05-08T08:26:16.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923877+0000", 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942837+0000", 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018959999000000002 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942917+0000", 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0519000000000006e-05 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943028+0000", 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111402 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50741 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.182626465999999, 2024-05-08T08:26:16.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019309865999999998, 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50741 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923891+0000", 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923894+0000", 2024-05-08T08:26:16.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1829999999999998e-06 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923902+0000", 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0910000000000004e-06 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923907+0000", 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6729999999999997e-06 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943059+0000", 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019151695 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943123+0000", 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4098999999999997e-05 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943194+0000", 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0970999999999995e-05 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50742 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.182595542000001, 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019467485, 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50742 2024-05-08T08:26:16.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923932+0000", 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6268000000000001e-05 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923939+0000", 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7270000000000007e-06 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923945+0000", 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5419999999999996e-06 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943232+0000", 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019287316999999998 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943315+0000", 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2854000000000003e-05 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943383+0000", 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7427000000000003e-05 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.946256553000001, 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:26:16.175 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:26:16.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:26:16.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.946235453, 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.180 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:26:16.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:26:16.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.946214794999999, 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.184 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:16.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.188 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:16.190 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:16.190 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:16.196+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:16.290 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:16.298+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:16.306 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:26:16.344 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:16.344 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:16.344 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:16.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:16.391 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:16.398+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:16.492 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:16.499+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:16.502 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:16.502 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:16.503 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:16.503 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:16.511 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:16.593 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:16.600+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:16.623 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:16.623 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:16.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:16.623 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69714 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.809036917, 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046246284999999998, 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69714 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.624 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820801+0000", 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-05-08T08:26:16.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820803+0000", 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820809+0000", 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7410000000000003e-06 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820815+0000", 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6200000000000004e-06 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866947+0000", 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046131923999999998 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866993+0000", 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867046+0000", 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3052e-05 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69715 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3162112~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.809014762, 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046369965999999999, 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.628 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69715 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820823+0000", 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820825+0000", 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.655e-06 2024-05-08T08:26:16.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820832+0000", 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0849999999999999e-06 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820838+0000", 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867075+0000", 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046236823000000003 2024-05-08T08:26:16.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867133+0000", 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867192+0000", 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9475999999999998e-05 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69716 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3391488~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808991261999999, 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046544169000000003, 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69716 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.633 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:16.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820848+0000", 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820855+0000", 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140000000000002e-06 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820861+0000", 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867234+0000", 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046372732 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867299+0000", 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5485000000000001e-05 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867390+0000", 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969000000000003e-05 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69717 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3620864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808969267, 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046712535999999999, 2024-05-08T08:26:16.637 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69717 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820870+0000", 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1320000000000001e-06 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820879+0000", 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820892+0000", 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3356000000000001e-05 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867422+0000", 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046529981999999998 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867485+0000", 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2614999999999996e-05 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867580+0000", 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5359000000000006e-05 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69718 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.641 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808936406000001, 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046863712000000002, 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69718 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:16.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820904+0000", 2024-05-08T08:26:16.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820911+0000", 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820917+0000", 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867609+0000", 2024-05-08T08:26:16.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046691734999999998 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867681+0000", 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1931e-05 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867764+0000", 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3356000000000006e-05 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69719 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4079616~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808912695, 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047037906999999997, 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69719 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820925+0000", 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820927+0000", 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820934+0000", 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.764e-06 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820939+0000", 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.553e-06 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867796+0000", 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046856496999999997 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867868+0000", 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2163999999999996e-05 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867962+0000", 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4337000000000002e-05 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.649 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69720 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4308992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808890549000001, 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047192222999999998, 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69720 2024-05-08T08:26:16.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:16.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820949+0000", 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820956+0000", 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820963+0000", 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-05-08T08:26:16.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867988+0000", 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047025083000000002 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868053+0000", 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4871999999999998e-05 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868139+0000", 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5626999999999999e-05 2024-05-08T08:26:16.653 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69721 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4538368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808866148, 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047345685999999998, 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.654 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69721 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820974+0000", 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820982+0000", 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8370000000000003e-06 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820988+0000", 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868167+0000", 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047179062000000001 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868232+0000", 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4889999999999997e-05 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868317+0000", 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4244000000000002e-05 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69722 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4767744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808841467000001, 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047510657999999997, 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.658 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69722 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821004+0000", 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821011+0000", 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6510000000000001e-06 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821019+0000", 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868347+0000", 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047327859 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868419+0000", 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2725999999999995e-05 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868506+0000", 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6793999999999996e-05 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69723 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4997120~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808809950000001, 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047669162000000001, 2024-05-08T08:26:16.662 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69723 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821028+0000", 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821030+0000", 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821037+0000", 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821043+0000", 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868536+0000", 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047493103000000002 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868611+0000", 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4647999999999994e-05 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868696+0000", 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5476999999999995e-05 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69724 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5226496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.666 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808786671, 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819591000000002, 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69724 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821051+0000", 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821053+0000", 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821060+0000", 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.097e-06 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821066+0000", 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868722+0000", 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047656233999999999 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868795+0000", 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2621999999999997e-05 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868870+0000", 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5352999999999995e-05 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.670 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69725 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5455872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808764378999999, 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047973017999999999, 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69725 2024-05-08T08:26:16.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:16.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821075+0000", 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.356e-06 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821082+0000", 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821088+0000", 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:26:16.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868907+0000", 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819382000000001 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868979+0000", 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1922999999999996e-05 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869046+0000", 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6724999999999996e-05 2024-05-08T08:26:16.674 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69726 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5685248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808724956000001, 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048091651999999999, 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.675 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69726 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821113+0000", 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821115+0000", 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7889999999999999e-06 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821124+0000", 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.191e-06 2024-05-08T08:26:16.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821130+0000", 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869065+0000", 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047935187999999997 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869129+0000", 2024-05-08T08:26:16.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4356999999999994e-05 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869204+0000", 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399000000000004e-05 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69727 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5914624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808698849000001, 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048288931, 2024-05-08T08:26:16.679 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69727 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821139+0000", 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821141+0000", 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821148+0000", 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821153+0000", 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869256+0000", 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048102180000000001 2024-05-08T08:26:16.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869343+0000", 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7828e-05 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869427+0000", 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3666999999999997e-05 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.683 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69728 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6144000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808676401, 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048487729, 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69728 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821164+0000", 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821171+0000", 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821176+0000", 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869489+0000", 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048312213999999999 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869566+0000", 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7657999999999994e-05 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869648+0000", 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2194000000000006e-05 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69729 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6373376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808653479, 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048680186, 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69729 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:16.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821186+0000", 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821192+0000", 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821197+0000", 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869699+0000", 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048501191999999999 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869782+0000", 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2973000000000007e-05 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869864+0000", 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2241000000000001e-05 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69730 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6602752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808633281000001, 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048837093999999998, 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69730 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821206+0000", 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821212+0000", 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821218+0000", 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869928+0000", 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048710003000000002 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869995+0000", 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7125000000000005e-05 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870041+0000", 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6134999999999998e-05 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69731 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6832128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808612209, 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048956353000000001, 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69731 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:16.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821228+0000", 2024-05-08T08:26:16.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821234+0000", 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821239+0000", 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.485e-06 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870066+0000", 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048826621000000001 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870133+0000", 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870181+0000", 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8553000000000001e-05 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.700 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69732 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7061504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808573320000001, 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049133148000000001, 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69732 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.701 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:16.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821267+0000", 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821286+0000", 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9122000000000001e-05 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821292+0000", 2024-05-08T08:26:16.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870231+0000", 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048938161000000001 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870316+0000", 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5143999999999997e-05 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870397+0000", 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1118000000000005e-05 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69733 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7290880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.808535995, 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049266982000000001, 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:16.705 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69733 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:16.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821302+0000", 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821304+0000", 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821311+0000", 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729999999999998e-06 2024-05-08T08:26:16.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821317+0000", 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2660000000000001e-06 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870426+0000", 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049109685 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870497+0000", 2024-05-08T08:26:16.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0860999999999998e-05 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870568+0000", 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0984999999999999e-05 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:16.709 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:16.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:16.711 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:16.718+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:16.811 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:16.819+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:16.861 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:16.862 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:16.862 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:16.862 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:16.870 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:16.912 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:16.919+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:17.013 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:17.021+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:17.021 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:17.021 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:17.022 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:17.022 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:17.030 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:17.052 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:17.114 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:17.121+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:17.142 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.423251245000003, 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:17.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.153557707999994, 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.124469941999998, 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.151 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 52.009318245999999, 2024-05-08T08:26:17.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:17.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:17.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:17.160 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.885671823999999, 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.161 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:17.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:17.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.885600676999999, 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:17.166 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:26:17.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:26:17.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.885254947, 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:17.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:17.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.885213657999998, 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:26:17.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21117 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 413696~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.671156408999998, 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003161652, 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21117 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477596+0000", 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8050000000000001e-06 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477599+0000", 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.616e-06 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477604+0000", 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477608+0000", 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480366+0000", 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027578020000000002 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480420+0000", 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3735e-05 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480756+0000", 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033637199999999998 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.184 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.671133550999997, 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:17.188 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21738 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 450560~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.524620394000003, 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031591710000000001, 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.189 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21738 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624133+0000", 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624140+0000", 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:26:17.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624145+0000", 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627084+0000", 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029395129999999999 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627161+0000", 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7096999999999997e-05 2024-05-08T08:26:17.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627290+0000", 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012866399999999999 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.378919721000003, 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:17.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.651717685000001, 2024-05-08T08:26:17.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:17.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:17.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.432737320000001, 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.203 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:17.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:17.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:17.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:17.207 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 38.944511955000003, 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:17.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:17.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:17.211 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 34.024247799999998, 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:17.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:17.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.876241665999999, 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:17.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:17.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:17.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.222 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.785380772, 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:17.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.4042505290000005, 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:26:17.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:26:17.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:91376 3.9 e76/18 3:97e339a7:::rbd_data.1191eeab3f85.0000000000000259:head v 76'288, mlcod=76'286)", 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 6.0986106529999997, 2024-05-08T08:26:17.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035121610000000002, 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:17.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050141+0000", 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.95e-07 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050150+0000", 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050168+0000", 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7751e-05 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050177+0000", 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7010000000000008e-06 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:17.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050242+0000", 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4276999999999995e-05 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050266+0000", 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4443e-05 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053607+0000", 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033412189999999999 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:17.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053653+0000", 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5287000000000002e-05 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:17.237 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:17.239 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:17.239 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:17.240 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:17.240 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:17.240 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:17.241 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:17.241 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:17.245+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:17.253 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:17.253 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:17.273 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:17.273 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:17.291 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:17.291 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:17.310 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:17.310 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:17.330 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:17.330 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:17.339 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:17.346+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:17.341 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone34 from 0xd678d3e (overlap 0x1c24f1a) to 0x2d07e9 2024-05-08T08:26:17.364 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:17.364 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:17.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:17.427 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-05-08T08:26:17.440 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:17.447+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:17.440 INFO:teuthology.orchestra.run.smithi031.stdout:5650 writesame 0x3b0bf94 thru 0x3b198c9 data_size 0x1(0xd936 bytes) 2024-05-08T08:26:17.480 INFO:teuthology.orchestra.run.smithi031.stdout:5652 trunc from 0xd678d3e to 0x3753618 2024-05-08T08:26:17.521 INFO:teuthology.orchestra.run.smithi031.stdout:5654 write 0x779f54d thru 0x77aa78d (0xb241 bytes) 2024-05-08T08:26:17.539 INFO:teuthology.orchestra.run.smithi031.stdout:5655 writesame 0xb42b50d thru 0xb43192c data_size 0x60(0x6420 bytes) 2024-05-08T08:26:17.539 INFO:teuthology.orchestra.run.smithi031.stdout:5656 read 0x77385ae thru 0x774038f (0x7de2 bytes) 2024-05-08T08:26:17.540 INFO:teuthology.orchestra.run.smithi031.stdout:5657 writesame 0x8528f21 thru 0x852bdd0 data_size 0x10(0x2eb0 bytes) 2024-05-08T08:26:17.540 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:17.547+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:17.543 INFO:teuthology.orchestra.run.smithi031.stdout:5658 read 0x9caf780 thru 0x9cbb0c5 (0xb946 bytes) 2024-05-08T08:26:17.543 INFO:teuthology.orchestra.run.smithi031.stdout:5659 write 0x9acaf81 thru 0x9ad2617 (0x7697 bytes) 2024-05-08T08:26:17.547 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:17.548 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:17.548 INFO:teuthology.orchestra.run.smithi031.stdout:5662 write 0xa0861b2 thru 0xa08d949 (0x7798 bytes) 2024-05-08T08:26:17.551 INFO:teuthology.orchestra.run.smithi031.stdout:5663 punch from 0x445724e to 0x445adf5, (0x3ba7 bytes) 2024-05-08T08:26:17.556 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:17.556 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:17.560 INFO:teuthology.orchestra.run.smithi031.stdout:5664 trunc from 0xb43192d to 0x3b8053c 2024-05-08T08:26:17.568 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:17.590 INFO:teuthology.orchestra.run.smithi031.stdout:5665 write 0xe702aa4 thru 0xe7046fe (0x1c5b bytes) 2024-05-08T08:26:17.597 INFO:teuthology.orchestra.run.smithi031.stdout:5666 read 0x9a49083 thru 0x9a51d66 (0x8ce4 bytes) 2024-05-08T08:26:17.606 INFO:teuthology.orchestra.run.smithi031.stdout:5667 trunc from 0xe7046ff to 0x6337b73 2024-05-08T08:26:17.618 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:17.627 INFO:teuthology.orchestra.run.smithi031.stdout:5669 write 0xa09042d thru 0xa09e7ae (0xe382 bytes) 2024-05-08T08:26:17.634 INFO:teuthology.orchestra.run.smithi031.stdout:5670 read 0x408e592 thru 0x409196b (0x33da bytes) 2024-05-08T08:26:17.634 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:17.634 INFO:teuthology.orchestra.run.smithi031.stdout:5673 punch from 0x3649d18 to 0x365461e, (0xa906 bytes) 2024-05-08T08:26:17.634 INFO:teuthology.orchestra.run.smithi031.stdout:5674 punch from 0x239ab3e to 0x239f79f, (0x4c61 bytes) 2024-05-08T08:26:17.634 INFO:teuthology.orchestra.run.smithi031.stdout:5676 read 0x7284430 thru 0x7292b71 (0xe742 bytes) 2024-05-08T08:26:17.634 INFO:teuthology.orchestra.run.smithi031.stdout:5677 read 0x749854 thru 0x7523f6 (0x8ba3 bytes) 2024-05-08T08:26:17.635 INFO:teuthology.orchestra.run.smithi031.stdout:5678 punch from 0x7b709f7 to 0x7b78d9a, (0x83a3 bytes) 2024-05-08T08:26:17.635 INFO:teuthology.orchestra.run.smithi031.stdout:5679 read 0x7fa066d thru 0x7fad3ef (0xcd83 bytes) 2024-05-08T08:26:17.635 INFO:teuthology.orchestra.run.smithi031.stdout:5680 read 0x47555ae thru 0x475be20 (0x6873 bytes) 2024-05-08T08:26:17.641 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:17.648+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:17.646 INFO:teuthology.orchestra.run.smithi031.stdout:5681 write 0xe195f6d thru 0xe1a14fb (0xb58f bytes) 2024-05-08T08:26:17.655 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:17.655 INFO:teuthology.orchestra.run.smithi031.stdout:5683 write 0x6d5018 thru 0x6d7fef (0x2fd8 bytes) 2024-05-08T08:26:17.662 INFO:teuthology.orchestra.run.smithi031.stdout:5684 trunc from 0xe1a14fc to 0xe70ef4b 2024-05-08T08:26:17.668 INFO:teuthology.orchestra.run.smithi031.stdout:5685 read 0xb2af82b thru 0xb2b14fe (0x1cd4 bytes) 2024-05-08T08:26:17.668 INFO:teuthology.orchestra.run.smithi031.stdout:5686 write 0x89458fa thru 0x8947447 (0x1b4e bytes) 2024-05-08T08:26:17.671 INFO:teuthology.orchestra.run.smithi031.stdout:5689 writesame 0xb919b90 thru 0xb9286e4 data_size 0x1(0xeb55 bytes) 2024-05-08T08:26:17.693 INFO:teuthology.orchestra.run.smithi031.stdout:5690 read 0x79dacf8 thru 0x79de54b (0x3854 bytes) 2024-05-08T08:26:17.693 INFO:teuthology.orchestra.run.smithi031.stdout:5692 read 0x52fa334 thru 0x5300977 (0x6644 bytes) 2024-05-08T08:26:17.693 INFO:teuthology.orchestra.run.smithi031.stdout:5694 clone 36 order 24 su 524288 sc 7 2024-05-08T08:26:17.714 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:17.714 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:17.724 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:17.764 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:17.772+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:17.830 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:17.830 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:17.839 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:17.882 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:17.889+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:17.943 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:17.949+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:17.944 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:17.945 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:17.953 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:18.044 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:18.052+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:18.099 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:18.099 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:18.108 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:18.145 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:18.153+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:18.213 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:18.214 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:18.222 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:18.266 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:18.273+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:18.321 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:26:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:18.330 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:18.340 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:18.381 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:18.388+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:18.447 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:18.454+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:18.487 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:18.487 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:18.496 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:18.548 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:18.555+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:18.642 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:18.642 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:18.648 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:18.656+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:18.651 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:18.750 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:18.757+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:18.756 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:18.756 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:18.765 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:18.851 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:18.857+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:18.872 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:18.872 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:18.881 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:18.965 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:18.972+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:19.028 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:19.028 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:19.037 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:19.078 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:19.085+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:19.153 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:19.160+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:19.184 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:19.184 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:19.193 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:19.253 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:19.260+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:19.300 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:19.301 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:19.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:19.315 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:26:19.338 INFO:teuthology.orchestra.run.smithi031.stdout:truncating image image_client.0-clone35 from 0xe70ef4b (overlap 0x3753618) to 0x16215a3 2024-05-08T08:26:19.348 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:19.360 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:19.367+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:19.455 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:19.462+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:19.479 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:19.479 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:19.491 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:19.531 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:19.532 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:19.554 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:19.554 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:19.555 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-05-08T08:26:19.555 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:19.562+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:19.562 INFO:teuthology.orchestra.run.smithi031.stdout:5695 punch from 0x43739bc to 0x437b15a, (0x779e bytes) 2024-05-08T08:26:19.565 INFO:teuthology.orchestra.run.smithi031.stdout:5696 writesame 0xc66c161 thru 0xc66ec93 data_size 0x1(0x2b33 bytes) 2024-05-08T08:26:19.574 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:19.574 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:19.580 INFO:teuthology.orchestra.run.smithi031.stdout:5697 trunc from 0xe70ef4b to 0x7f1a2fb 2024-05-08T08:26:19.595 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:19.595 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:19.601 INFO:teuthology.orchestra.run.smithi031.stdout:5698 read 0x7f5d29 thru 0x7fc528 (0x6800 bytes) 2024-05-08T08:26:19.601 INFO:teuthology.orchestra.run.smithi031.stdout:5699 write 0x8cd9a4 thru 0x8d16bb (0x3d18 bytes) 2024-05-08T08:26:19.625 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:19.625 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:19.642 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:19.642 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:19.657 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:19.664+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:19.657 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:19.657 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:19.658 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:19.658 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:19.660 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:19.660 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:19.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:19.678 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:19.679 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:19.758 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:19.765+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50476 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 161792~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880499824000001, 2024-05-08T08:26:19.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018840722000000001, 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50476 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915326+0000", 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915327+0000", 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915329+0000", 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3630000000000001e-06 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915336+0000", 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4880000000000001e-06 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915342+0000", 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.933988+0000", 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018646593 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934072+0000", 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3626000000000002e-05 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934167+0000", 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5087999999999995e-05 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50477 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 171008~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880477352, 2024-05-08T08:26:19.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019055283999999999, 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50477 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915349+0000", 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915351+0000", 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.311e-06 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915363+0000", 2024-05-08T08:26:19.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2031000000000001e-05 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915369+0000", 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6930000000000004e-06 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934223+0000", 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018853741 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934307+0000", 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4154999999999996e-05 2024-05-08T08:26:19.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934404+0000", 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6973999999999997e-05 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50478 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 180224~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880448732000001, 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019257972000000002, 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50478 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915377+0000", 2024-05-08T08:26:19.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915378+0000", 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915387+0000", 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0999999999999993e-06 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915396+0000", 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.2499999999999995e-06 2024-05-08T08:26:19.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915401+0000", 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.259e-06 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934434+0000", 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019032970999999999 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934535+0000", 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100479 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934635+0000", 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000100445 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50479 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 189440~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880416473, 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019428513000000001, 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50479 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915410+0000", 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7399999999999998e-07 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915413+0000", 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915419+0000", 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7320000000000001e-06 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915424+0000", 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0799999999999996e-06 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934664+0000", 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019239903999999999 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934740+0000", 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5833000000000004e-05 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934838+0000", 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7832999999999997e-05 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50480 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 198656~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880394561999999, 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019651075, 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50480 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915431+0000", 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915432+0000", 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915434+0000", 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.221e-06 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915440+0000", 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4770000000000002e-06 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915444+0000", 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8330000000000001e-06 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934880+0000", 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019435555 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.934984+0000", 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010403099999999999 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935083+0000", 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.8566000000000005e-05 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50481 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 207872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880374624000002, 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019849193000000001, 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50481 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915451+0000", 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915452+0000", 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915454+0000", 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3719999999999999e-06 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915461+0000", 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6290000000000004e-06 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915466+0000", 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1699999999999996e-06 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935126+0000", 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019660074 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935221+0000", 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4598000000000005e-05 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935301+0000", 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.996e-05 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50482 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 217088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880353045, 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020069704000000001, 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.813 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50482 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915473+0000", 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915476+0000", 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1770000000000001e-06 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915482+0000", 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4030000000000004e-06 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915487+0000", 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9219999999999996e-06 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935343+0000", 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019856492999999999 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935433+0000", 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9485000000000001e-05 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935543+0000", 2024-05-08T08:26:19.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109832 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50483 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 226304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880332719999998, 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020296027000000001, 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.817 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50483 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915493+0000", 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915494+0000", 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5600000000000001e-07 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915496+0000", 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1239999999999999e-06 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915502+0000", 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7939999999999999e-06 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915507+0000", 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.074e-06 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935582+0000", 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020075790999999999 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935672+0000", 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0074999999999994e-05 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935789+0000", 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011681299999999999 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50484 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 235520~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880312861, 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020571613999999998, 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.821 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50484 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915513+0000", 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915514+0000", 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915516+0000", 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2809999999999998e-06 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915649+0000", 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000132776 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915668+0000", 2024-05-08T08:26:19.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9253e-05 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935846+0000", 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020177820999999999 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.935950+0000", 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000104478 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936085+0000", 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000134618 2024-05-08T08:26:19.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50485 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 244736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880146175, 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020672174000000001, 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50485 2024-05-08T08:26:19.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915680+0000", 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4700000000000001e-07 2024-05-08T08:26:19.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915684+0000", 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5820000000000002e-06 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915692+0000", 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1829999999999994e-06 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915706+0000", 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3747999999999999e-05 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936120+0000", 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020414472999999999 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936205+0000", 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4561000000000006e-05 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936352+0000", 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014698000000000001 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.828 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50486 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 253952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.880110297000002, 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020885167, 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50486 2024-05-08T08:26:19.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:19.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915716+0000", 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6400000000000003e-07 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915724+0000", 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4200000000000001e-06 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915734+0000", 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0796e-05 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.915748+0000", 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3318e-05 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936388+0000", 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.020640642000000001 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936491+0000", 2024-05-08T08:26:19.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010300400000000001 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.936601+0000", 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000109523 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50737 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 208896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.872157389000002, 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018673419, 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.833 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50737 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923669+0000", 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923672+0000", 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.252e-06 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923679+0000", 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5880000000000003e-06 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923737+0000", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8035000000000003e-05 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942158+0000", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018421334000000001 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942240+0000", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1409000000000006e-05 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942342+0000", 2024-05-08T08:26:19.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000102252 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50738 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 218112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.872074041000001, 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018832728, 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.837 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50738 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923752+0000", 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923753+0000", 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9699999999999996e-07 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923756+0000", 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4510000000000001e-06 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923766+0000", 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.049e-05 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923773+0000", 2024-05-08T08:26:19.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.054e-06 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942387+0000", 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018614117999999999 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942475+0000", 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7874999999999995e-05 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942585+0000", 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011014300000000001 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50739 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 227328~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.872009077000001, 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018990561999999999, 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.841 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50739 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:19.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923817+0000", 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2499999999999995e-07 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923828+0000", 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0988999999999999e-05 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923837+0000", 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6889999999999993e-06 2024-05-08T08:26:19.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923851+0000", 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3528e-05 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942618+0000", 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018766998999999999 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942704+0000", 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5995000000000006e-05 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942807+0000", 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000103837 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50740 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 236544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.871965693, 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019168166, 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50740 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923860+0000", 2024-05-08T08:26:19.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923861+0000", 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5200000000000002e-07 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923863+0000", 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5069999999999999e-06 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923871+0000", 2024-05-08T08:26:19.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0569999999999993e-06 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923877+0000", 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2299999999999999e-06 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942837+0000", 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.018959999000000002 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.942917+0000", 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0519000000000006e-05 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943028+0000", 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111402 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50741 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 245760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.871941844999998, 2024-05-08T08:26:19.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019309865999999998, 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50741 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923884+0000", 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923891+0000", 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1539999999999996e-06 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923894+0000", 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1829999999999998e-06 2024-05-08T08:26:19.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923902+0000", 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0910000000000004e-06 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923907+0000", 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6729999999999997e-06 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943059+0000", 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019151695 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943123+0000", 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4098999999999997e-05 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943194+0000", 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0970999999999995e-05 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:50742 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 254976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.871910921000001, 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019467485, 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 50742 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923915+0000", 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:26:19.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923932+0000", 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6268000000000001e-05 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923939+0000", 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7270000000000007e-06 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.923945+0000", 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5419999999999996e-06 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943232+0000", 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.019287316999999998 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943315+0000", 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2854000000000003e-05 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:51.943383+0000", 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7427000000000003e-05 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.635571932000001, 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:26:19.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.635550832, 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:26:19.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:19.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:26:19.865 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.635530173999999, 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:19.866 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:19.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:26:19.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.870 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:19.870 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:19.870 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:19.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:19.871 INFO:teuthology.orchestra.run.smithi031.stdout:5700 read 0x791eb1e thru 0x7921c33 (0x3116 bytes) 2024-05-08T08:26:19.872 INFO:teuthology.orchestra.run.smithi031.stdout:5702 write 0x6340739 thru 0x634bfa3 (0xb86b bytes) 2024-05-08T08:26:19.872 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:19.878+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:19.912 INFO:teuthology.orchestra.run.smithi031.stdout:5703 write 0xdcfdf95 thru 0xdd04daf (0x6e1b bytes) 2024-05-08T08:26:19.926 INFO:teuthology.orchestra.run.smithi031.stdout:5704 trunc from 0xdd04db0 to 0x24de2f1 2024-05-08T08:26:19.972 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:19.978+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:20.054 INFO:teuthology.orchestra.run.smithi031.stdout:5705 read 0x4bd39a thru 0x4c9662 (0xc2c9 bytes) 2024-05-08T08:26:20.064 INFO:teuthology.orchestra.run.smithi031.stdout:5706 trunc from 0x24de2f1 to 0xa7006ec 2024-05-08T08:26:20.067 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:20.067 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:20.067 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:20.067 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:20.069 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.073 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:20.080+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:20.073 INFO:teuthology.orchestra.run.smithi031.stdout:5708 trunc from 0xa7006ec to 0x6494150 2024-05-08T08:26:20.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:20.084 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.084 INFO:teuthology.orchestra.run.smithi031.stdout:5711 read 0x48a5a1a thru 0x48b23e9 (0xc9d0 bytes) 2024-05-08T08:26:20.084 INFO:teuthology.orchestra.run.smithi031.stdout:5712 writesame 0x15cad14 thru 0x15cdc7c data_size 0x1(0x2f69 bytes) 2024-05-08T08:26:20.090 INFO:teuthology.orchestra.run.smithi031.stdout:5713 read 0x21a2c0d thru 0x21a9263 (0x6657 bytes) 2024-05-08T08:26:20.090 INFO:teuthology.orchestra.run.smithi031.stdout:5715 read 0xca66f6 thru 0xca76ea (0xff5 bytes) 2024-05-08T08:26:20.091 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.091 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.101 INFO:teuthology.orchestra.run.smithi031.stdout:5718 writesame 0xb629dd4 thru 0xb629fc9 data_size 0x1(0x1f6 bytes) 2024-05-08T08:26:20.101 INFO:teuthology.orchestra.run.smithi031.stdout:5720 read 0x136cbde thru 0x1375ac5 (0x8ee8 bytes) 2024-05-08T08:26:20.102 INFO:teuthology.orchestra.run.smithi031.stdout:5721 read 0x305588d thru 0x3056cca (0x143e bytes) 2024-05-08T08:26:20.102 INFO:teuthology.orchestra.run.smithi031.stdout:5722 writesame 0xa545864 thru 0xa55544b data_size 0xc(0xfbe8 bytes) 2024-05-08T08:26:20.109 INFO:teuthology.orchestra.run.smithi031.stdout:5723 read 0x43e900a thru 0x43f4867 (0xb85e bytes) 2024-05-08T08:26:20.110 INFO:teuthology.orchestra.run.smithi031.stdout:5725 write 0x1d7629 thru 0x1e51c2 (0xdb9a bytes) 2024-05-08T08:26:20.120 INFO:teuthology.orchestra.run.smithi031.stdout:5726 writesame 0xd9e6121 thru 0xd9ed33c data_size 0x4(0x721c bytes) 2024-05-08T08:26:20.121 INFO:teuthology.orchestra.run.smithi031.stdout:5727 writesame 0x8538eab thru 0x853f9dc data_size 0x1(0x6b32 bytes) 2024-05-08T08:26:20.143 INFO:teuthology.orchestra.run.smithi031.stdout:5730 trunc from 0xd9ed33d to 0x80f291f 2024-05-08T08:26:20.158 INFO:teuthology.orchestra.run.smithi031.stdout:5733 trunc from 0x80f291f to 0x659c02 2024-05-08T08:26:20.174 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:20.181+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:20.180 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.180 INFO:teuthology.orchestra.run.smithi031.stdout:5735 read 0x59f09 thru 0x60714 (0x680c bytes) 2024-05-08T08:26:20.180 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.191 INFO:teuthology.orchestra.run.smithi031.stdout:5739 write 0x492ec2f thru 0x49331a0 (0x4572 bytes) 2024-05-08T08:26:20.192 INFO:teuthology.orchestra.run.smithi031.stdout:5741 read 0x1b827be thru 0x1b83bc6 (0x1409 bytes) 2024-05-08T08:26:20.196 INFO:teuthology.orchestra.run.smithi031.stdout:5743 trunc from 0x49331a1 to 0x760ef2e 2024-05-08T08:26:20.202 INFO:teuthology.orchestra.run.smithi031.stdout:5744 trunc from 0x760ef2e to 0x6685bf2 2024-05-08T08:26:20.204 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:20.204 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:20.204 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:20.211 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.211 INFO:teuthology.orchestra.run.smithi031.stdout:5746 trunc from 0x6685bf2 to 0x6a55ff3 2024-05-08T08:26:20.213 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:20.217 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.218 INFO:teuthology.orchestra.run.smithi031.stdout:5748 writesame 0x4bf1543 thru 0x4bffc24 data_size 0x3(0xe6e2 bytes) 2024-05-08T08:26:20.227 INFO:teuthology.orchestra.run.smithi031.stdout:5749 read 0xf135f7 thru 0xf1675c (0x3166 bytes) 2024-05-08T08:26:20.242 INFO:teuthology.orchestra.run.smithi031.stdout:5751 writesame 0xd7a7fae thru 0xd7aae4b data_size 0x4e(0x2e9e bytes) 2024-05-08T08:26:20.242 INFO:teuthology.orchestra.run.smithi031.stdout:5752 write 0x5cc62bf thru 0x5cd3d4d (0xda8f bytes) 2024-05-08T08:26:20.244 INFO:teuthology.orchestra.run.smithi031.stdout:5754 punch from 0xb932147 to 0xb93c55f, (0xa418 bytes) 2024-05-08T08:26:20.244 INFO:teuthology.orchestra.run.smithi031.stdout:5755 punch from 0xccd4562 to 0xcce04d7, (0xbf75 bytes) 2024-05-08T08:26:20.244 INFO:teuthology.orchestra.run.smithi031.stdout:5757 writesame 0x2085c6f thru 0x208a854 data_size 0x2(0x4be6 bytes) 2024-05-08T08:26:20.249 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.249 INFO:teuthology.orchestra.run.smithi031.stdout:5760 writesame 0x1ee9abe thru 0x1eed949 data_size 0x2(0x3e8c bytes) 2024-05-08T08:26:20.253 INFO:teuthology.orchestra.run.smithi031.stdout:5763 write 0xbc1f9bb thru 0xbc2efcd (0xf613 bytes) 2024-05-08T08:26:20.256 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.257 INFO:teuthology.orchestra.run.smithi031.stdout:5765 write 0x588e94d thru 0x58933fc (0x4ab0 bytes) 2024-05-08T08:26:20.268 INFO:teuthology.orchestra.run.smithi031.stdout:5767 write 0xe7718ae thru 0xe773a71 (0x21c4 bytes) 2024-05-08T08:26:20.270 INFO:teuthology.orchestra.run.smithi031.stdout:5768 write 0x7e997ae thru 0x7ea0101 (0x6954 bytes) 2024-05-08T08:26:20.274 INFO:teuthology.orchestra.run.smithi031.stdout:5770 punch from 0x79ff716 to 0x7a0ed71, (0xf65b bytes) 2024-05-08T08:26:20.274 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.275 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:20.281+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:20.279 INFO:teuthology.orchestra.run.smithi031.stdout:5772 writesame 0xecb9179 thru 0xecbb6b9 data_size 0x21(0x2541 bytes) 2024-05-08T08:26:20.279 INFO:teuthology.orchestra.run.smithi031.stdout:5776 write 0x3335b5b thru 0x3338e65 (0x330b bytes) 2024-05-08T08:26:20.281 INFO:teuthology.orchestra.run.smithi031.stdout:5778 read 0x17eaaa0 thru 0x17f32ae (0x880f bytes) 2024-05-08T08:26:20.281 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.282 INFO:teuthology.orchestra.run.smithi031.stdout:5781 write 0x6214ac9 thru 0x621e298 (0x97d0 bytes) 2024-05-08T08:26:20.283 INFO:teuthology.orchestra.run.smithi031.stdout:5782 read 0x5157933 thru 0x5159950 (0x201e bytes) 2024-05-08T08:26:20.283 INFO:teuthology.orchestra.run.smithi031.stdout:5784 read 0xc948a56 thru 0xc94b97d (0x2f28 bytes) 2024-05-08T08:26:20.287 INFO:teuthology.orchestra.run.smithi031.stdout:5786 trunc from 0xecbb6ba to 0xb716231 2024-05-08T08:26:20.303 INFO:teuthology.orchestra.run.smithi031.stdout:5788 write 0x3d08246 thru 0x3d10c69 (0x8a24 bytes) 2024-05-08T08:26:20.304 INFO:teuthology.orchestra.run.smithi031.stdout:5790 write 0x5feb49 thru 0x6011e9 (0x26a1 bytes) 2024-05-08T08:26:20.306 INFO:teuthology.orchestra.run.smithi031.stdout:5791 read 0x1016ae0 thru 0x101db96 (0x70b7 bytes) 2024-05-08T08:26:20.306 INFO:teuthology.orchestra.run.smithi031.stdout:5792 read 0x7332ef6 thru 0x733f589 (0xc694 bytes) 2024-05-08T08:26:20.306 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.307 INFO:teuthology.orchestra.run.smithi031.stdout:5794 trunc from 0xb716231 to 0xc823c2f 2024-05-08T08:26:20.313 INFO:teuthology.orchestra.run.smithi031.stdout:5797 read 0x6483c7e thru 0x648aafd (0x6e80 bytes) 2024-05-08T08:26:20.317 INFO:teuthology.orchestra.run.smithi031.stdout:5799 trunc from 0xc823c2f to 0x91ba35e 2024-05-08T08:26:20.322 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:20.322 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:20.323 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-05-08T08:26:20.323 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-05-08T08:26:20.324 INFO:teuthology.orchestra.run.smithi031.stdout:5801 flatten 2024-05-08T08:26:20.330 INFO:teuthology.orchestra.run.smithi031.stdout:5802 write 0x776e1a9 thru 0x776f36e (0x11c6 bytes) 2024-05-08T08:26:20.334 INFO:teuthology.orchestra.run.smithi031.stdout:5805 punch from 0x608a96f to 0x6095434, (0xaac5 bytes) 2024-05-08T08:26:20.334 INFO:teuthology.orchestra.run.smithi031.stdout:5806 writesame 0x2e8136a thru 0x2e8341d data_size 0x1c(0x20b4 bytes) 2024-05-08T08:26:20.335 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69714 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524828101000001, 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046246284999999998, 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69714 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820801+0000", 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820803+0000", 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820809+0000", 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7410000000000003e-06 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820815+0000", 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6200000000000004e-06 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866947+0000", 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046131923999999998 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866993+0000", 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867046+0000", 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3052e-05 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69715 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3162112~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524805946000001, 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046369965999999999, 2024-05-08T08:26:20.343 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69715 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:20.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820823+0000", 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820825+0000", 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.655e-06 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820832+0000", 2024-05-08T08:26:20.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0849999999999999e-06 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820838+0000", 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867075+0000", 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046236823000000003 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867133+0000", 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:26:20.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867192+0000", 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9475999999999998e-05 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69716 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3391488~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524782446, 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046544169000000003, 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69716 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:20.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820848+0000", 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820855+0000", 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140000000000002e-06 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820861+0000", 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867234+0000", 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046372732 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867299+0000", 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5485000000000001e-05 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867390+0000", 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969000000000003e-05 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69717 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3620864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524760450999999, 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046712535999999999, 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69717 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:20.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820870+0000", 2024-05-08T08:26:20.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1320000000000001e-06 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820879+0000", 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820892+0000", 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3356000000000001e-05 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867422+0000", 2024-05-08T08:26:20.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046529981999999998 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867485+0000", 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2614999999999996e-05 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867580+0000", 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5359000000000006e-05 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69718 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524727590000001, 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046863712000000002, 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69718 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:20.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820904+0000", 2024-05-08T08:26:20.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820911+0000", 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820917+0000", 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867609+0000", 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046691734999999998 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867681+0000", 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1931e-05 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867764+0000", 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3356000000000006e-05 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69719 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4079616~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524703879, 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047037906999999997, 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69719 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:20.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820925+0000", 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820927+0000", 2024-05-08T08:26:20.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820934+0000", 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.764e-06 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820939+0000", 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.553e-06 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867796+0000", 2024-05-08T08:26:20.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046856496999999997 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867868+0000", 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2163999999999996e-05 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867962+0000", 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4337000000000002e-05 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69720 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4308992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524681733000001, 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047192222999999998, 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69720 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820949+0000", 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820956+0000", 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820963+0000", 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867988+0000", 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047025083000000002 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868053+0000", 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4871999999999998e-05 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868139+0000", 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5626999999999999e-05 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69721 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4538368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524657332, 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047345685999999998, 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69721 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820974+0000", 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820982+0000", 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8370000000000003e-06 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820988+0000", 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868167+0000", 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047179062000000001 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868232+0000", 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4889999999999997e-05 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868317+0000", 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4244000000000002e-05 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69722 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4767744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524632651000001, 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047510657999999997, 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69722 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821004+0000", 2024-05-08T08:26:20.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821011+0000", 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6510000000000001e-06 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821019+0000", 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868347+0000", 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047327859 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868419+0000", 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2725999999999995e-05 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868506+0000", 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6793999999999996e-05 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69723 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4997120~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:20.376 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524601134000001, 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047669162000000001, 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69723 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821028+0000", 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821030+0000", 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821037+0000", 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821043+0000", 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868536+0000", 2024-05-08T08:26:20.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047493103000000002 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868611+0000", 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4647999999999994e-05 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868696+0000", 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5476999999999995e-05 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69724 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5226496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524577855, 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819591000000002, 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69724 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821051+0000", 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821053+0000", 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821060+0000", 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.097e-06 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821066+0000", 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868722+0000", 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047656233999999999 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868795+0000", 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2621999999999997e-05 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868870+0000", 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5352999999999995e-05 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69725 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5455872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524555563, 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047973017999999999, 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69725 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:20.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821075+0000", 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.356e-06 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821082+0000", 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821088+0000", 2024-05-08T08:26:20.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868907+0000", 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819382000000001 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868979+0000", 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1922999999999996e-05 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869046+0000", 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6724999999999996e-05 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69726 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5685248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524516139999999, 2024-05-08T08:26:20.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048091651999999999, 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69726 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821113+0000", 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821115+0000", 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7889999999999999e-06 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821124+0000", 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.191e-06 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821130+0000", 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869065+0000", 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047935187999999997 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869129+0000", 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4356999999999994e-05 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869204+0000", 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399000000000004e-05 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69727 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5914624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524490032999999, 2024-05-08T08:26:20.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048288931, 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69727 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821139+0000", 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821141+0000", 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821148+0000", 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821153+0000", 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869256+0000", 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048102180000000001 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869343+0000", 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7828e-05 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869427+0000", 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3666999999999997e-05 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69728 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6144000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524467585, 2024-05-08T08:26:20.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048487729, 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69728 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821164+0000", 2024-05-08T08:26:20.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821171+0000", 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821176+0000", 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869489+0000", 2024-05-08T08:26:20.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048312213999999999 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869566+0000", 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7657999999999994e-05 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869648+0000", 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2194000000000006e-05 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.401 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69729 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6373376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524444663000001, 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048680186, 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69729 2024-05-08T08:26:20.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-05-08T08:26:20.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821186+0000", 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821192+0000", 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821197+0000", 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869699+0000", 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048501191999999999 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869782+0000", 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2973000000000007e-05 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869864+0000", 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2241000000000001e-05 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69730 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6602752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524424464999999, 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048837093999999998, 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69730 2024-05-08T08:26:20.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:20.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821206+0000", 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821212+0000", 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821218+0000", 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869928+0000", 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048710003000000002 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869995+0000", 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7125000000000005e-05 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870041+0000", 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6134999999999998e-05 2024-05-08T08:26:20.409 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69731 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6832128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524403393, 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048956353000000001, 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.410 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69731 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821228+0000", 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821234+0000", 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:26:20.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821239+0000", 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.485e-06 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870066+0000", 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048826621000000001 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870133+0000", 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870181+0000", 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8553000000000001e-05 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69732 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7061504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524364504000001, 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049133148000000001, 2024-05-08T08:26:20.414 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69732 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821267+0000", 2024-05-08T08:26:20.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821286+0000", 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9122000000000001e-05 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821292+0000", 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870231+0000", 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048938161000000001 2024-05-08T08:26:20.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870316+0000", 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5143999999999997e-05 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870397+0000", 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1118000000000005e-05 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69733 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7290880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.524327179, 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049266982000000001, 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69733 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821302+0000", 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821304+0000", 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821311+0000", 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729999999999998e-06 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821317+0000", 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2660000000000001e-06 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870426+0000", 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049109685 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870497+0000", 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0860999999999998e-05 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870568+0000", 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0984999999999999e-05 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.423 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:20.424 INFO:teuthology.orchestra.run.smithi031.stdout:5809 write 0xac9595e thru 0xac9cf69 (0x760c bytes) 2024-05-08T08:26:20.424 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:5810 write 0xd42ffa7 thru 0xd4389e1 (0x8a3b bytes) 2024-05-08T08:26:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:5811 writesame 0x5e89680 thru 0x5e8bce9 data_size 0xb(0x266a bytes) 2024-05-08T08:26:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:5812 write 0x73bd419 thru 0x73c7812 (0xa3fa bytes) 2024-05-08T08:26:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:5813 trunc from 0xd4389e2 to 0x7a20b5a 2024-05-08T08:26:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:5814 write 0x644ce58 thru 0x6451fc0 (0x5169 bytes) 2024-05-08T08:26:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:5815 trunc from 0x7a20b5a to 0x62d3c9c 2024-05-08T08:26:20.426 INFO:teuthology.orchestra.run.smithi031.stdout:5816 writesame 0xbc6a4b1 thru 0xbc72f5e data_size 0x61(0x8aae bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5819 read 0x91d84b1 thru 0x91dd4c8 (0x5018 bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5820 punch from 0x9ccd0ca to 0x9cd0864, (0x379a bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5821 punch from 0xa3f2f17 to 0xa3f969f, (0x6788 bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5822 read 0x23ef1e3 thru 0x23f87cf (0x95ed bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5823 read 0x57bb560 thru 0x57bf186 (0x3c27 bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5824 read 0x2a46f47 thru 0x2a56158 (0xf212 bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5830 punch from 0x911636b to 0x91261d6, (0xfe6b bytes) 2024-05-08T08:26:20.427 INFO:teuthology.orchestra.run.smithi031.stdout:5831 writesame 0xa1ac1d6 thru 0xa1b5fba data_size 0x1(0x9de5 bytes) 2024-05-08T08:26:20.427 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:20.431+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:20.428 INFO:teuthology.orchestra.run.smithi031.stdout:5832 writesame 0x71fe095 thru 0x71fe7e1 data_size 0x7(0x74d bytes) 2024-05-08T08:26:20.428 INFO:teuthology.orchestra.run.smithi031.stdout:5833 read 0x8eb231e thru 0x8eb7eee (0x5bd1 bytes) 2024-05-08T08:26:20.428 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.428 INFO:teuthology.orchestra.run.smithi031.stdout:5835 punch from 0x7752ea1 to 0x775b4cb, (0x862a bytes) 2024-05-08T08:26:20.431 INFO:teuthology.orchestra.run.smithi031.stdout:5836 trunc from 0xbc72f5f to 0x9496989 2024-05-08T08:26:20.443 INFO:teuthology.orchestra.run.smithi031.stdout:5837 trunc from 0x9496989 to 0x6065286 2024-05-08T08:26:20.458 INFO:teuthology.orchestra.run.smithi031.stdout:5838 punch from 0x28fc154 to 0x29025e1, (0x648d bytes) 2024-05-08T08:26:20.460 INFO:teuthology.orchestra.run.smithi031.stdout:5841 trunc from 0x6065286 to 0x8166245 2024-05-08T08:26:20.466 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.466 INFO:teuthology.orchestra.run.smithi031.stdout:5844 read 0x5f3ad4c thru 0x5f46c04 (0xbeb9 bytes) 2024-05-08T08:26:20.468 INFO:teuthology.orchestra.run.smithi031.stdout:5845 trunc from 0x8166245 to 0x671e13b 2024-05-08T08:26:20.486 INFO:teuthology.orchestra.run.smithi031.stdout:5846 write 0xa8c78e6 thru 0xa8ccec7 (0x55e2 bytes) 2024-05-08T08:26:20.490 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.490 INFO:teuthology.orchestra.run.smithi031.stdout:5848 read 0xa13a940 thru 0xa13afb5 (0x676 bytes) 2024-05-08T08:26:20.490 INFO:teuthology.orchestra.run.smithi031.stdout:5849 read 0x607aebc thru 0x6088895 (0xd9da bytes) 2024-05-08T08:26:20.490 INFO:teuthology.orchestra.run.smithi031.stdout:5852 write 0x7530e27 thru 0x753d1a1 (0xc37b bytes) 2024-05-08T08:26:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:20.494 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:20.495 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:20.498 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:20.498 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:20.498 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-05-08T08:26:20.499 INFO:teuthology.orchestra.run.smithi031.stdout:5853 write 0xaa47469 thru 0xaa4d360 (0x5ef8 bytes) 2024-05-08T08:26:20.502 INFO:teuthology.orchestra.run.smithi031.stdout:5854 read 0x3698fb3 thru 0x36a12a3 (0x82f1 bytes) 2024-05-08T08:26:20.506 INFO:teuthology.orchestra.run.smithi031.stdout:5855 trunc from 0xaa4d361 to 0x77b1bcb 2024-05-08T08:26:20.513 INFO:teuthology.orchestra.run.smithi031.stdout:5857 writesame 0x2a883e0 thru 0x2a8ef01 data_size 0x15(0x6b22 bytes) 2024-05-08T08:26:20.515 INFO:teuthology.orchestra.run.smithi031.stdout:5858 read 0x685de18 thru 0x686d463 (0xf64c bytes) 2024-05-08T08:26:20.515 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.524 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:20.532+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:20.529 INFO:teuthology.orchestra.run.smithi031.stdout:5863 write 0xd57f53a thru 0xd5867c7 (0x728e bytes) 2024-05-08T08:26:20.533 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.536 INFO:teuthology.orchestra.run.smithi031.stdout:5868 trunc from 0xd5867c8 to 0xb1045f5 2024-05-08T08:26:20.543 INFO:teuthology.orchestra.run.smithi031.stdout:5869 punch from 0x9fefb96 to 0x9ff4dfe, (0x5268 bytes) 2024-05-08T08:26:20.543 INFO:teuthology.orchestra.run.smithi031.stdout:5870 punch from 0x33bbaf9 to 0x33c439b, (0x88a2 bytes) 2024-05-08T08:26:20.543 INFO:teuthology.orchestra.run.smithi031.stdout:5872 write 0x6c6690d thru 0x6c74c7d (0xe371 bytes) 2024-05-08T08:26:20.547 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.547 INFO:teuthology.orchestra.run.smithi031.stdout:5876 punch from 0x215fb96 to 0x2160953, (0xdbd bytes) 2024-05-08T08:26:20.547 INFO:teuthology.orchestra.run.smithi031.stdout:5877 write 0x43cab8c thru 0x43d439d (0x9812 bytes) 2024-05-08T08:26:20.549 INFO:teuthology.orchestra.run.smithi031.stdout:5878 read 0x7dbbcd3 thru 0x7dc6e4c (0xb17a bytes) 2024-05-08T08:26:20.549 INFO:teuthology.orchestra.run.smithi031.stdout:5879 read 0x89f7f83 thru 0x8a0398a (0xba08 bytes) 2024-05-08T08:26:20.549 INFO:teuthology.orchestra.run.smithi031.stdout:5880 read 0x9066986 thru 0x9071edf (0xb55a bytes) 2024-05-08T08:26:20.550 INFO:teuthology.orchestra.run.smithi031.stdout:5881 write 0x697b0b3 thru 0x697fd9a (0x4ce8 bytes) 2024-05-08T08:26:20.551 INFO:teuthology.orchestra.run.smithi031.stdout:5882 read 0xa07936a thru 0xa07c0ed (0x2d84 bytes) 2024-05-08T08:26:20.551 INFO:teuthology.orchestra.run.smithi031.stdout:5883 read 0x15be791 thru 0x15cb839 (0xd0a9 bytes) 2024-05-08T08:26:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:5885 read 0x355a7a8 thru 0x3565c1a (0xb473 bytes) 2024-05-08T08:26:20.552 INFO:teuthology.orchestra.run.smithi031.stdout:5886 read 0xaa76266 thru 0xaa8516c (0xef07 bytes) 2024-05-08T08:26:20.557 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:20.557 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:20.557 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:20.557 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:20.562 INFO:teuthology.orchestra.run.smithi031.stdout:5887 trunc from 0xb1045f5 to 0x26db627 2024-05-08T08:26:20.571 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:20.588 INFO:teuthology.orchestra.run.smithi031.stdout:5888 write 0x9b123a8 thru 0x9b161a4 (0x3dfd bytes) 2024-05-08T08:26:20.591 INFO:teuthology.orchestra.run.smithi031.stdout:5889 read 0x7e351c0 thru 0x7e3aad2 (0x5913 bytes) 2024-05-08T08:26:20.591 INFO:teuthology.orchestra.run.smithi031.stdout:5890 read 0x732b0db thru 0x733201a (0x6f40 bytes) 2024-05-08T08:26:20.591 INFO:teuthology.orchestra.run.smithi031.stdout:5892 punch from 0x1b99a40 to 0x1b9a9fd, (0xfbd bytes) 2024-05-08T08:26:20.593 INFO:teuthology.orchestra.run.smithi031.stdout:5893 trunc from 0x9b161a5 to 0x7a71d1a 2024-05-08T08:26:20.600 INFO:teuthology.orchestra.run.smithi031.stdout:5894 writesame 0x484355a thru 0x484fa15 data_size 0xc(0xc4bc bytes) 2024-05-08T08:26:20.602 INFO:teuthology.orchestra.run.smithi031.stdout:5895 write 0x30ca130 thru 0x30d3860 (0x9731 bytes) 2024-05-08T08:26:20.604 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.604 INFO:teuthology.orchestra.run.smithi031.stdout:5897 write 0x4db4bd3 thru 0x4db7e4b (0x3279 bytes) 2024-05-08T08:26:20.606 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.613 INFO:teuthology.orchestra.run.smithi031.stdout:5899 trunc from 0x7a71d1a to 0x1b56690 2024-05-08T08:26:20.625 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:20.633+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:20.625 INFO:teuthology.orchestra.run.smithi031.stdout:5901 read 0x193c320 thru 0x19431fb (0x6edc bytes) 2024-05-08T08:26:20.626 INFO:teuthology.orchestra.run.smithi031.stdout:5902 punch from 0x13543ce to 0x135dfee, (0x9c20 bytes) 2024-05-08T08:26:20.626 INFO:teuthology.orchestra.run.smithi031.stdout:5903 punch from 0xbd52db to 0xbe2341, (0xd066 bytes) 2024-05-08T08:26:20.637 INFO:teuthology.orchestra.run.smithi031.stdout:5904 writesame 0x6edaa33 thru 0x6ee5205 data_size 0x1(0xa7d3 bytes) 2024-05-08T08:26:20.639 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:20.645 INFO:teuthology.orchestra.run.smithi031.stdout:5907 trunc from 0x6ee5206 to 0x1231cbd 2024-05-08T08:26:20.673 INFO:teuthology.orchestra.run.smithi031.stdout:5908 write 0xec8f6c9 thru 0xec95a97 (0x63cf bytes) 2024-05-08T08:26:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:20.676 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:20.677 INFO:teuthology.orchestra.run.smithi031.stdout:5909 read 0xea20d4d thru 0xea24524 (0x37d8 bytes) 2024-05-08T08:26:20.677 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:20.678 INFO:teuthology.orchestra.run.smithi031.stdout:5910 writesame 0x2f802eb thru 0x2f8fa52 data_size 0xa8(0xf768 bytes) 2024-05-08T08:26:20.678 INFO:teuthology.orchestra.run.smithi031.stdout:5911 write 0x9bf0c06 thru 0x9bfd8ce (0xccc9 bytes) 2024-05-08T08:26:20.679 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:20.680 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:20.680 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-05-08T08:26:20.681 INFO:teuthology.orchestra.run.smithi031.stdout:5915 punch from 0xbc3b426 to 0xbc42be0, (0x77ba bytes) 2024-05-08T08:26:20.695 INFO:teuthology.orchestra.run.smithi031.stdout:5916 trunc from 0xec95a98 to 0x24ccf2d 2024-05-08T08:26:20.726 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:20.733+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:20.727 INFO:teuthology.orchestra.run.smithi031.stdout:5917 write 0xc7ac490 thru 0xc7b32f0 (0x6e61 bytes) 2024-05-08T08:26:20.730 INFO:teuthology.orchestra.run.smithi031.stdout:5920 write 0x8e683bf thru 0x8e72975 (0xa5b7 bytes) 2024-05-08T08:26:20.730 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:20.731 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:20.731 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:20.733 INFO:teuthology.orchestra.run.smithi031.stdout:5921 read 0x1954b2a thru 0x1962a01 (0xded8 bytes) 2024-05-08T08:26:20.734 INFO:teuthology.orchestra.run.smithi031.stdout:5922 punch from 0x90ac2bf to 0x90ada50, (0x1791 bytes) 2024-05-08T08:26:20.734 INFO:teuthology.orchestra.run.smithi031.stdout:5923 write 0x90d57d6 thru 0x90ddb10 (0x833b bytes) 2024-05-08T08:26:20.737 INFO:teuthology.orchestra.run.smithi031.stdout:5924 clone 37 order 25 su 16777216 sc 14 2024-05-08T08:26:20.741 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:20.827 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:20.834+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:20.829 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:20.832 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:20.832 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:20.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-05-08T08:26:20.893 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.174945332999997, 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 68.905251796000002, 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.898 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:20.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.901 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.876164029999998, 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.902 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:26:20.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:26:20.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.906 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.761012334, 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.907 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:20.908 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.909 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.910 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.911 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.637365912, 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.912 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:20.913 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:20.914 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:20.915 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.637294765, 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.916 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:20.917 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:26:20.918 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:26:20.919 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.920 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.636949035000001, 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.921 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.922 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:20.923 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.636907745999999, 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:26:20.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21117 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 413696~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.422850496999999, 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003161652, 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21117 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.929 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477596+0000", 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8050000000000001e-06 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477599+0000", 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.616e-06 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477604+0000", 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477608+0000", 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480366+0000", 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027578020000000002 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480420+0000", 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3735e-05 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480756+0000", 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033637199999999998 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.422827638999998, 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.933 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.936 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21738 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 450560~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.276314481999997, 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031591710000000001, 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21738 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624133+0000", 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624140+0000", 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624145+0000", 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627084+0000", 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029395129999999999 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627161+0000", 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7096999999999997e-05 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627290+0000", 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012866399999999999 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.130613809000003, 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:20.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.944 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.403411773000002, 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.184431408000002, 2024-05-08T08:26:20.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:20.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.696206042999997, 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:20.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:20.957 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.775941887999998, 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.627935753999999, 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:20.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.537074859999997, 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:20.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:20.972 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 10.155944616999999, 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:26:20.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:91376 3.9 e76/18 3:97e339a7:::rbd_data.1191eeab3f85.0000000000000259:head v 76'288, mlcod=76'286)", 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.8503047410000004, 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035121610000000002, 2024-05-08T08:26:20.976 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050141+0000", 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.95e-07 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050150+0000", 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050168+0000", 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7751e-05 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050177+0000", 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7010000000000008e-06 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050242+0000", 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4276999999999995e-05 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050266+0000", 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4443e-05 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053607+0000", 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033412189999999999 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053653+0000", 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5287000000000002e-05 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:20.980 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:20.982 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:20.982 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:20.989+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:21.012 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:21.014 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:21.015 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:21.015 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-05-08T08:26:21.082 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:21.089+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:21.123 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:21.123 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:21.123 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:21.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:21.183 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:21.190+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:21.195 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:21.196 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:21.198 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:21.199 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:21.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-05-08T08:26:21.244 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:21.244 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:21.244 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:21.244 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:21.244 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:21.253 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:21.296 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:21.304+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:21.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:21.342 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:21.343 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:21.345 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:21.345 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:21.346 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-05-08T08:26:21.372 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.372 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218815408000001, 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:26:21.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218805323000002, 2024-05-08T08:26:21.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:26:21.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218794844000001, 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.381 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:21.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:26:21.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:26:21.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218777918000001, 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.385 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218756998, 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:26:21.389 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:21.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:26:21.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218718682999999, 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.394 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.395 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:26:21.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218694936999999, 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.398 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:21.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:26:21.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:26:21.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218672093999999, 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:21.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:26:21.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218647704999999, 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:26:21.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218625785, 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.412 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:26:21.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.414 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218604550999999, 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:26:21.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:26:21.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.418 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.218583107000001, 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.217598803000001, 2024-05-08T08:26:21.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:26:21.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:26:21.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.426 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.217577601999999, 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:26:21.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.217566132000002, 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:26:21.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:21.434 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.217556372000001, 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:26:21.437 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.217546583000001, 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.439 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.440 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.441 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.442 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.217536059, 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:26:21.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:26:21.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.217526378999999, 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.447 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.448 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.9942652299999999, 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:26:21.451 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.452 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:26:21.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:26:21.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:21.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:21.456 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:21.457 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:21.458 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:21.465+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:21.545 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-05-08T08:26:21.558 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:21.565+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110289 3.1c 3.3af799bc (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577419+0000", 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0086467100000000002, 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086721109999999997, 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.582 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110289 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577419+0000", 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577419+0000", 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577419+0000", 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-07 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577420+0000", 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.13e-06 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577423+0000", 2024-05-08T08:26:21.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9629999999999998e-06 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577429+0000", 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6570000000000002e-06 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110414 3.1a 3.8b46a65a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578019+0000", 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0080458060000000008, 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0081354760000000009, 2024-05-08T08:26:21.585 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110414 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578019+0000", 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578019+0000", 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578020+0000", 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2100000000000002e-07 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578022+0000", 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8959999999999999e-06 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578027+0000", 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2050000000000001e-06 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578035+0000", 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7449999999999995e-06 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110290 3.1c 3.3af799bc (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577453+0000", 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0086119669999999999, 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087684549999999997, 2024-05-08T08:26:21.588 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110290 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577453+0000", 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577453+0000", 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577454+0000", 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4300000000000003e-07 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577456+0000", 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:21.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577462+0000", 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6069999999999998e-06 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577467+0000", 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8559999999999998e-06 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110415 3.1a 3.8b46a65a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578041+0000", 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0080245690000000005, 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082350700000000006, 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110415 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.592 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578041+0000", 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578041+0000", 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578041+0000", 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578047+0000", 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.558e-06 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578052+0000", 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4809999999999997e-06 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578057+0000", 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4440000000000001e-06 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110292 3.1c 3.3af799bc (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577473+0000", 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0085920409999999999, 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088346659999999997, 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.595 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110292 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577473+0000", 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577473+0000", 2024-05-08T08:26:21.596 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577474+0000", 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577475+0000", 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8470000000000001e-06 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577481+0000", 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.015e-06 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577486+0000", 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7770000000000002e-06 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110416 3.1a 3.8b46a65a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578063+0000", 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0080022700000000006, 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0082860999999999994, 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110416 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.599 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578063+0000", 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578063+0000", 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.600 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578063+0000", 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.53e-07 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578065+0000", 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9309999999999998e-06 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578070+0000", 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5589999999999999e-06 2024-05-08T08:26:21.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578072+0000", 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5359999999999999e-06 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110293 3.1c 3.3af799bc (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577492+0000", 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0085734279999999993, 2024-05-08T08:26:21.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089099899999999996, 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110293 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577492+0000", 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577492+0000", 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577492+0000", 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.68e-07 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577494+0000", 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.874e-06 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577500+0000", 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5210000000000002e-06 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577504+0000", 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5569999999999997e-06 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110417 3.1a 3.8b46a65a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578076+0000", 2024-05-08T08:26:21.605 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0079895350000000007, 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0083804340000000008, 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110417 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578076+0000", 2024-05-08T08:26:21.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578076+0000", 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578076+0000", 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2700000000000001e-07 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578077+0000", 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1999999999999999e-06 2024-05-08T08:26:21.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578081+0000", 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5200000000000002e-06 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578083+0000", 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4830000000000002e-06 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110294 3.1c 3.3af799bc (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.608 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577510+0000", 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0085553369999999997, 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0090040060000000002, 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110294 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577510+0000", 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577510+0000", 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577510+0000", 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4000000000000002e-07 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577518+0000", 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6809999999999998e-06 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577524+0000", 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8250000000000001e-06 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577529+0000", 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7339999999999999e-06 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.611 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110418 3.1a 3.8b46a65a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578086+0000", 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0079790369999999996, 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0084832680000000004, 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110418 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.612 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578086+0000", 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578086+0000", 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578087+0000", 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9499999999999998e-07 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578088+0000", 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1030000000000001e-06 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578091+0000", 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578093+0000", 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4509999999999999e-06 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110295 3.1c 3.3af799bc (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577535+0000", 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0085300759999999993, 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0090629979999999992, 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110295 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577535+0000", 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577535+0000", 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577536+0000", 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577538+0000", 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.069e-06 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577582+0000", 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3946999999999998e-05 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577587+0000", 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5690000000000004e-06 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.617 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110419 3.1a 3.8b46a65a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578096+0000", 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0079692879999999997, 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0085474980000000006, 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110419 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578096+0000", 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578096+0000", 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578096+0000", 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0200000000000001e-07 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578101+0000", 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8400000000000002e-06 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578104+0000", 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0790000000000001e-06 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578106+0000", 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3360000000000002e-06 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.620 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110296 3.1c 3.3af799bc (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577595+0000", 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0084701110000000007, 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0090766500000000003, 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.621 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110296 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577595+0000", 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577595+0000", 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577596+0000", 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.15e-07 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577598+0000", 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.255e-06 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577603+0000", 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.48e-06 2024-05-08T08:26:21.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577608+0000", 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6940000000000003e-06 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110420 3.1a 3.8b46a65a (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578110+0000", 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0079557659999999995, 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086050450000000004, 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.624 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110420 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578110+0000", 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578110+0000", 2024-05-08T08:26:21.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578110+0000", 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2399999999999999e-07 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578111+0000", 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.19e-06 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578115+0000", 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9500000000000003e-06 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578117+0000", 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3520000000000001e-06 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110429 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578120+0000", 2024-05-08T08:26:21.627 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0079451959999999999, 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086521979999999998, 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110429 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578120+0000", 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578120+0000", 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578120+0000", 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578121+0000", 2024-05-08T08:26:21.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.11e-06 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578124+0000", 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8909999999999999e-06 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578127+0000", 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3769999999999999e-06 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110430 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578130+0000", 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0079356770000000004, 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0086968140000000006, 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110430 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578130+0000", 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578130+0000", 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578130+0000", 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4699999999999998e-07 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578131+0000", 2024-05-08T08:26:21.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.9000000000000005e-07 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578134+0000", 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8480000000000001e-06 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578137+0000", 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.633 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110353 3.1a 3:5a6562d1:::rbd_data.11918d3cc87b.0000000000000001:25 [read 851968~16384 out=16384b] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577665+0000", 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0084004430000000005, 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0092137839999999992, 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110353 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577665+0000", 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577665+0000", 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577665+0000", 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577667+0000", 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:26:21.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577673+0000", 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2580000000000003e-06 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577677+0000", 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4290000000000003e-06 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.586713+0000", 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0090358590000000003 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.586790+0000", 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6594000000000006e-05 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.586879+0000", 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9029000000000004e-05 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110431 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578152+0000", 2024-05-08T08:26:21.637 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.007913191, 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0087766919999999991, 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110431 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578152+0000", 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578152+0000", 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578152+0000", 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578154+0000", 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5990000000000001e-06 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578158+0000", 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4979999999999998e-06 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578160+0000", 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5629999999999999e-06 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110355 3.1a 3:5a6562d1:::rbd_data.11918d3cc87b.0000000000000001:25 [read 884736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.640 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577687+0000", 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0083782600000000002, 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0092782019999999993, 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "reached pg", 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110355 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577687+0000", 2024-05-08T08:26:21.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577687+0000", 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577687+0000", 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9000000000000002e-07 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577690+0000", 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.159e-06 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577694+0000", 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1869999999999999e-06 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577696+0000", 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.661e-06 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:21.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.586900+0000", 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0092033340000000005 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110432 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578165+0000", 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0079000890000000008, 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0088343929999999994, 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.644 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110432 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578165+0000", 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578165+0000", 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.645 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578166+0000", 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2599999999999998e-07 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578167+0000", 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0410000000000001e-06 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578169+0000", 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9229999999999998e-06 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578172+0000", 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5189999999999999e-06 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110357 3.1a 3.8b46a65a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577700+0000", 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0083648249999999993, 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0093375379999999994, 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.647 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110357 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577700+0000", 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.648 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577700+0000", 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577701+0000", 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0600000000000001e-07 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577702+0000", 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1680000000000001e-06 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.649 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577705+0000", 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4970000000000001e-06 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577709+0000", 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2710000000000001e-06 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110433 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578175+0000", 2024-05-08T08:26:21.650 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0078902509999999992, 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0089159430000000008, 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110433 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578175+0000", 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578175+0000", 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578175+0000", 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1300000000000001e-07 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578176+0000", 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.018e-06 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578179+0000", 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7630000000000001e-06 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578182+0000", 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3980000000000002e-06 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110359 3.1a 3.8b46a65a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577712+0000", 2024-05-08T08:26:21.653 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0083533510000000002, 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.009449397, 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110359 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577712+0000", 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577712+0000", 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577712+0000", 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.11e-07 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577713+0000", 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.026e-06 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577716+0000", 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9840000000000001e-06 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577719+0000", 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4380000000000002e-06 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110434 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578184+0000", 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0078809520000000001, 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0090342310000000002, 2024-05-08T08:26:21.656 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110434 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578184+0000", 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578184+0000", 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578185+0000", 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8900000000000001e-07 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578186+0000", 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2600000000000001e-07 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578188+0000", 2024-05-08T08:26:21.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8159999999999998e-06 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578191+0000", 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5040000000000001e-06 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110361 3.1a 3.8b46a65a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577722+0000", 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0083436719999999999, 2024-05-08T08:26:21.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0095354470000000007, 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110361 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577722+0000", 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577722+0000", 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577722+0000", 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577723+0000", 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0079999999999999e-06 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.661 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577726+0000", 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9000000000000002e-06 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577728+0000", 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4650000000000001e-06 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110435 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578194+0000", 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.00787151, 2024-05-08T08:26:21.662 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0091033030000000001, 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110435 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.663 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578194+0000", 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578194+0000", 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578194+0000", 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.664 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578199+0000", 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2240000000000003e-06 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578204+0000", 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5680000000000001e-06 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.665 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578208+0000", 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3340000000000003e-06 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110363 3.1a 3.8b46a65a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577731+0000", 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0083338030000000007, 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0095934000000000002, 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.666 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110363 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577731+0000", 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.667 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577731+0000", 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577732+0000", 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9999999999999999e-07 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577733+0000", 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6299999999999993e-07 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577736+0000", 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7340000000000002e-06 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577742+0000", 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.665e-06 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110436 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578214+0000", 2024-05-08T08:26:21.669 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0078507730000000001, 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.009137849, 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110436 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578214+0000", 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578214+0000", 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578215+0000", 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4899999999999995e-07 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578217+0000", 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9410000000000001e-06 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578223+0000", 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5450000000000003e-06 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578229+0000", 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.793e-06 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110365 3.1a 3.8b46a65a (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.672 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577745+0000", 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0083202090000000003, 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0096542850000000003, 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110365 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577745+0000", 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577745+0000", 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577745+0000", 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.23e-07 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577751+0000", 2024-05-08T08:26:21.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5729999999999998e-06 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577754+0000", 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2899999999999998e-06 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577757+0000", 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6410000000000002e-06 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110437 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578236+0000", 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.00782974, 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0092301550000000003, 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110437 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.676 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578236+0000", 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578236+0000", 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578236+0000", 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3799999999999998e-07 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578238+0000", 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.77e-06 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578243+0000", 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6130000000000003e-06 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.678 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578248+0000", 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4190000000000004e-06 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110374 3.3 3.c0e820c3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577760+0000", 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0083053329999999998, 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0097658420000000003, 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110374 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577760+0000", 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577760+0000", 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577760+0000", 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1899999999999999e-07 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577761+0000", 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2300000000000001e-06 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577765+0000", 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8430000000000003e-06 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577768+0000", 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3980000000000002e-06 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110438 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578253+0000", 2024-05-08T08:26:21.682 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0078118450000000004, 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0093322390000000009, 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110438 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578253+0000", 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578253+0000", 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578254+0000", 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2399999999999999e-07 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578260+0000", 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5810000000000002e-06 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578267+0000", 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.46e-06 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578271+0000", 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6920000000000001e-06 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110375 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577771+0000", 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0082939439999999993, 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098468970000000003, 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110375 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577771+0000", 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577771+0000", 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577772+0000", 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1199999999999999e-07 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.687 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577773+0000", 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0690000000000001e-06 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577776+0000", 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.551e-06 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577779+0000", 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7209999999999999e-06 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110439 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578278+0000", 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.007787609, 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0093740660000000003, 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110439 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578278+0000", 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578278+0000", 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578278+0000", 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.39e-07 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578280+0000", 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.001e-06 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578286+0000", 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6840000000000001e-06 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578290+0000", 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999998e-06 2024-05-08T08:26:21.691 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110376 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577782+0000", 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0082831659999999998, 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0099576909999999994, 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110376 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577782+0000", 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577782+0000", 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577782+0000", 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.36e-07 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577783+0000", 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.95e-07 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577786+0000", 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0800000000000002e-06 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577789+0000", 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.554e-06 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110440 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578297+0000", 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0077685289999999997, 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0095098660000000005, 2024-05-08T08:26:21.695 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110440 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578297+0000", 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578297+0000", 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578297+0000", 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7599999999999997e-07 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578299+0000", 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.86e-06 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578305+0000", 2024-05-08T08:26:21.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5500000000000002e-06 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578309+0000", 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4100000000000001e-06 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110377 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577807+0000", 2024-05-08T08:26:21.698 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0082586859999999995, 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010058891, 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110377 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577807+0000", 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577807+0000", 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577807+0000", 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577809+0000", 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5230000000000001e-06 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577812+0000", 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4000000000000001e-06 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577814+0000", 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.57e-06 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.701 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110441 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578315+0000", 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0077506839999999999, 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0096050969999999999, 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110441 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.702 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578315+0000", 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578315+0000", 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578315+0000", 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6800000000000001e-07 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578317+0000", 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8220000000000001e-06 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578323+0000", 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0650000000000004e-06 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578327+0000", 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2169999999999996e-06 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.704 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110378 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577818+0000", 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0082473700000000004, 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010151944, 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110378 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.705 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577818+0000", 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577818+0000", 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577818+0000", 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2800000000000003e-07 2024-05-08T08:26:21.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577819+0000", 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.048e-06 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577822+0000", 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7779999999999999e-06 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577824+0000", 2024-05-08T08:26:21.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110442 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578334+0000", 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0077317210000000004, 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0096686970000000004, 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.708 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110442 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578334+0000", 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578334+0000", 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578334+0000", 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8000000000000002e-07 2024-05-08T08:26:21.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578336+0000", 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6950000000000001e-06 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578341+0000", 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7710000000000003e-06 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578346+0000", 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4630000000000003e-06 2024-05-08T08:26:21.710 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110379 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577828+0000", 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0082377270000000002, 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010240361, 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110379 2024-05-08T08:26:21.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577828+0000", 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577828+0000", 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577828+0000", 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4999999999999998e-07 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577832+0000", 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2699999999999998e-06 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577836+0000", 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3859999999999999e-06 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577838+0000", 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4839999999999998e-06 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.713 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110443 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578352+0000", 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0077137669999999998, 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0097713599999999998, 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110443 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578352+0000", 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578352+0000", 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578352+0000", 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3299999999999998e-07 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578354+0000", 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9069999999999999e-06 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578359+0000", 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6930000000000004e-06 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578364+0000", 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3100000000000002e-06 2024-05-08T08:26:21.716 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110380 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577841+0000", 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0082239549999999998, 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010351412000000001, 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110380 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.717 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577841+0000", 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577841+0000", 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577842+0000", 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.41e-07 2024-05-08T08:26:21.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577843+0000", 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0920000000000001e-06 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577846+0000", 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8540000000000001e-06 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577848+0000", 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4229999999999999e-06 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110444 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578369+0000", 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0076961549999999997, 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0098650460000000006, 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110444 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578369+0000", 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578369+0000", 2024-05-08T08:26:21.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578369+0000", 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1100000000000002e-07 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578371+0000", 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.849e-06 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578377+0000", 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4399999999999996e-06 2024-05-08T08:26:21.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578381+0000", 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-06 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110381 3.3 3.c0e820c3 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577851+0000", 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0082144339999999996, 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010420382000000001, 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.723 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110381 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577851+0000", 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577851+0000", 2024-05-08T08:26:21.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577851+0000", 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4699999999999998e-07 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577852+0000", 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5099999999999998e-07 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577856+0000", 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2420000000000002e-06 2024-05-08T08:26:21.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577861+0000", 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3030000000000001e-06 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110387 3.1c 3:3d99ef5c:::rbd_data.11918d3cc87b.0000000000000004:25 [read 950272~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577963+0000", 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0081020869999999991, 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010367213, 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.726 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "reached pg", 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110387 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577963+0000", 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577963+0000", 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577964+0000", 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9100000000000004e-07 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577966+0000", 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9300000000000002e-06 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577971+0000", 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5419999999999996e-06 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577976+0000", 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7550000000000002e-06 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.588276+0000", 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.01030039 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110388 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.577981+0000", 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0080840479999999999, 2024-05-08T08:26:21.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010414725999999999, 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110388 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577981+0000", 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577981+0000", 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577982+0000", 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577983+0000", 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.734e-06 2024-05-08T08:26:21.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577988+0000", 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.152e-06 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.577994+0000", 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1540000000000001e-06 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:110389 3.1c 3.3af799bc (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e85)", 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:21.578000+0000", 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.0080657460000000004, 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010450823, 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "queued for pg", 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 110389 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:21.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578000+0000", 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578000+0000", 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578000+0000", 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6199999999999998e-07 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578002+0000", 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9700000000000002e-06 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578008+0000", 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6489999999999996e-06 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:21.578012+0000", 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.656e-06 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.735 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:21.736 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.736 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:21.736 INFO:teuthology.orchestra.run.smithi149.stdout: ], 2024-05-08T08:26:21.736 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 48 2024-05-08T08:26:21.736 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:21.736 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:21.737 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:21.738 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:21.738 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:21.743+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:21.739 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:21.740 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:21.740 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-05-08T08:26:21.741 INFO:teuthology.orchestra.run.smithi031.stdout:5925 trunc from 0xc7b32f1 to 0xb1ae597 2024-05-08T08:26:21.741 INFO:teuthology.orchestra.run.smithi031.stdout:5926 read 0x18eff7b thru 0x18f8c91 (0x8d17 bytes) 2024-05-08T08:26:21.741 INFO:teuthology.orchestra.run.smithi031.stdout:5927 trunc from 0xb1ae597 to 0xb549da8 2024-05-08T08:26:21.741 INFO:teuthology.orchestra.run.smithi031.stdout:5928 write 0xa714744 thru 0xa7226bc (0xdf79 bytes) 2024-05-08T08:26:21.741 INFO:teuthology.orchestra.run.smithi031.stdout:5929 writesame 0x9df2fbf thru 0x9df8e80 data_size 0x2(0x5ec2 bytes) 2024-05-08T08:26:21.741 INFO:teuthology.orchestra.run.smithi031.stdout:5931 read 0x4ca203c thru 0x4ca5c1f (0x3be4 bytes) 2024-05-08T08:26:21.741 INFO:teuthology.orchestra.run.smithi031.stdout:5933 write 0x55ccc5f thru 0x55d1582 (0x4924 bytes) 2024-05-08T08:26:21.742 INFO:teuthology.orchestra.run.smithi031.stdout:5936 writesame 0xb8c98e7 thru 0xb8d5bb0 data_size 0x3(0xc2ca bytes) 2024-05-08T08:26:21.742 INFO:teuthology.orchestra.run.smithi031.stdout:5937 trunc from 0xb8d5bb1 to 0x4d3ca82 2024-05-08T08:26:21.742 INFO:teuthology.orchestra.run.smithi031.stdout:5938 writesame 0x183a529 thru 0x18475e9 data_size 0x1(0xd0c1 bytes) 2024-05-08T08:26:21.773 INFO:teuthology.orchestra.run.smithi031.stdout:5940 write 0xa09d704 thru 0xa0aa4cc (0xcdc9 bytes) 2024-05-08T08:26:21.783 INFO:teuthology.orchestra.run.smithi031.stdout:5941 writesame 0xae92cd7 thru 0xaea0048 data_size 0x5(0xd372 bytes) 2024-05-08T08:26:21.785 INFO:teuthology.orchestra.run.smithi031.stdout:5942 read 0x66ecd67 thru 0x66f64d1 (0x976b bytes) 2024-05-08T08:26:21.785 INFO:teuthology.orchestra.run.smithi031.stdout:5943 read 0x4167e20 thru 0x4168c27 (0xe08 bytes) 2024-05-08T08:26:21.791 INFO:teuthology.orchestra.run.smithi031.stdout:5944 writesame 0xaed2071 thru 0xaed2f74 data_size 0x3e(0xf04 bytes) 2024-05-08T08:26:21.793 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:21.793 INFO:teuthology.orchestra.run.smithi031.stdout:5947 punch from 0xf948b1 to 0xfa35a7, (0xecf6 bytes) 2024-05-08T08:26:21.793 INFO:teuthology.orchestra.run.smithi031.stdout:5948 punch from 0x5bd2faa to 0x5bdc15b, (0x91b1 bytes) 2024-05-08T08:26:21.804 INFO:teuthology.orchestra.run.smithi031.stdout:5950 write 0xed2d891 thru 0xed35ae1 (0x8251 bytes) 2024-05-08T08:26:21.819 INFO:teuthology.orchestra.run.smithi031.stdout:5951 write 0xc2382fc thru 0xc241dc2 (0x9ac7 bytes) 2024-05-08T08:26:21.823 INFO:teuthology.orchestra.run.smithi031.stdout:5952 write 0x759d040 thru 0x759f65f (0x2620 bytes) 2024-05-08T08:26:21.826 INFO:teuthology.orchestra.run.smithi031.stdout:5954 write 0xe664d84 thru 0xe672a24 (0xdca1 bytes) 2024-05-08T08:26:21.829 INFO:teuthology.orchestra.run.smithi031.stdout:5957 read 0x4778242 thru 0x477938a (0x1149 bytes) 2024-05-08T08:26:21.829 INFO:teuthology.orchestra.run.smithi031.stdout:5958 read 0xa69d1b3 thru 0xa6a3a76 (0x68c4 bytes) 2024-05-08T08:26:21.829 INFO:teuthology.orchestra.run.smithi031.stdout:5959 read 0x271a432 thru 0x271c35c (0x1f2b bytes) 2024-05-08T08:26:21.830 INFO:teuthology.orchestra.run.smithi031.stdout:5960 write 0x2b99e2e thru 0x2ba8270 (0xe443 bytes) 2024-05-08T08:26:21.836 INFO:teuthology.orchestra.run.smithi031.stdout:5961 read 0x6ab1fbc thru 0x6ab28a7 (0x8ec bytes) 2024-05-08T08:26:21.838 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:21.845+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:21.849 INFO:teuthology.orchestra.run.smithi031.stdout:5963 trunc from 0xed35ae2 to 0x4b9036e 2024-05-08T08:26:21.864 INFO:teuthology.orchestra.run.smithi031.stdout:5965 read 0x3420e5c thru 0x34298f4 (0x8a99 bytes) 2024-05-08T08:26:21.877 INFO:teuthology.orchestra.run.smithi031.stdout:5966 trunc from 0x4b9036e to 0xe7ea0a3 2024-05-08T08:26:21.883 INFO:teuthology.orchestra.run.smithi031.stdout:5967 read 0x9b4ad1f thru 0x9b5344b (0x872d bytes) 2024-05-08T08:26:21.883 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:21.883 INFO:teuthology.orchestra.run.smithi031.stdout:5970 read 0x581d2a1 thru 0x5821523 (0x4283 bytes) 2024-05-08T08:26:21.883 INFO:teuthology.orchestra.run.smithi031.stdout:5971 read 0xc2c1232 thru 0xc2ca782 (0x9551 bytes) 2024-05-08T08:26:21.883 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:21.883 INFO:teuthology.orchestra.run.smithi031.stdout:5975 read 0x427d130 thru 0x4288aee (0xb9bf bytes) 2024-05-08T08:26:21.883 INFO:teuthology.orchestra.run.smithi031.stdout:5977 write 0x4008a7 thru 0x40252e (0x1c88 bytes) 2024-05-08T08:26:21.885 INFO:teuthology.orchestra.run.smithi031.stdout:5978 write 0x1407bd3 thru 0x1414e86 (0xd2b4 bytes) 2024-05-08T08:26:21.886 INFO:teuthology.orchestra.run.smithi031.stdout:5979 read 0xe492499 thru 0xe49d44d (0xafb5 bytes) 2024-05-08T08:26:21.887 INFO:teuthology.orchestra.run.smithi031.stdout:5980 write 0x8f5819f thru 0x8f5cc03 (0x4a65 bytes) 2024-05-08T08:26:21.890 INFO:teuthology.orchestra.run.smithi031.stdout:5982 read 0x26b56a8 thru 0x26c01a0 (0xaaf9 bytes) 2024-05-08T08:26:21.891 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:21.891 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:21.891 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:21.891 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:21.891 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:21.891 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:21.891 INFO:teuthology.orchestra.run.smithi031.stdout:5985 read 0xb7e6ed1 thru 0xb7f229b (0xb3cb bytes) 2024-05-08T08:26:21.893 INFO:teuthology.orchestra.run.smithi031.stdout:5986 trunc from 0xe7ea0a3 to 0xca5e67d 2024-05-08T08:26:21.900 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:21.907 INFO:teuthology.orchestra.run.smithi031.stdout:5989 trunc from 0xca5e67d to 0x676f19f 2024-05-08T08:26:21.922 INFO:teuthology.orchestra.run.smithi031.stdout:5990 trunc from 0x676f19f to 0x2f232d4 2024-05-08T08:26:21.923 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:21.923 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:21.939 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:21.947+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:21.944 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:21.945 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:no valid command found; 10 closest matches: 2024-05-08T08:26:21.946 INFO:teuthology.orchestra.run.smithi031.stderr:0 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:1 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:2 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:abort 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:assert 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:bench [] [] [] [] 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs debug_inject_read_zeros 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs files list 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:bluefs stats 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:bluestore allocator dump block 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stderr:Error EINVAL: invalid command 2024-05-08T08:26:21.947 INFO:teuthology.orchestra.run.smithi031.stdout:5991 write 0xa06f579 thru 0xa07dcec (0xe774 bytes) 2024-05-08T08:26:21.950 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:26:21.950 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-05-08T08:26:21.950 INFO:teuthology.orchestra.run.smithi031.stdout:5993 trunc from 0xa07dced to 0x98c4ce2 2024-05-08T08:26:21.963 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:21.964 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:21.964 INFO:teuthology.orchestra.run.smithi031.stdout:5994 trunc from 0x98c4ce2 to 0xc6f8dba 2024-05-08T08:26:21.970 INFO:teuthology.orchestra.run.smithi031.stdout:compare and write disabled 2024-05-08T08:26:21.970 INFO:teuthology.orchestra.run.smithi031.stdout:5997 write 0x926e204 thru 0x927bfbc (0xddb9 bytes) 2024-05-08T08:26:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:5998 read 0x4f5436f thru 0x4f60a1e (0xc6b0 bytes) 2024-05-08T08:26:21.974 INFO:teuthology.orchestra.run.smithi031.stdout:5999 write 0x407e1d5 thru 0x40888ae (0xa6da bytes) 2024-05-08T08:26:21.981 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:21.982 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:21.990 INFO:teuthology.orchestra.run.smithi031.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-05-08T08:26:22.000 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:22.000 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:22.020 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:22.020 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:22.040 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:22.040 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:22.040 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:22.047+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:22.051 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.051 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:22.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:22.058 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:22.061 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:22.144 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:22.151+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:22.206 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.206 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:22.215 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:22.241 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:22.249+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:22.338 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:26:22.343 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:22.350+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:22.373 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.373 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:22.383 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:22.443 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:22.451+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:22.498 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.498 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:22.507 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:22.550 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:22.557+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:22.612 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.612 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:22.621 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:22.645 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:22.653+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:22.747 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:22.755+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:22.766 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.766 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:22.775 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:22.848 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:22.855+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:22.879 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.879 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:22.888 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:22.948 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:22.956+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:22.993 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:22.993 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.002 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:23.048 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:23.055+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:23.107 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:23.107 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.116 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:23.157 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:23.164+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:23.250 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:23.257+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:23.261 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:23.261 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:23.351 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:23.358+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:23.403 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.414 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:23.451 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:23.458+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:23.552 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:23.559+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:23.566 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:23.566 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.575 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:23.653 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:23.660+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:23.685 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:23.685 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.694 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:23.754 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:23.761+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:23.804 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:23.804 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.813 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:23.855 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:23.862+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:23.956 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:23.963+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:23.962 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:23.962 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:23.971 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:24.056 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:24.064+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:24.071 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:24.081 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:24.090 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:24.175 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:24.183+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:24.199 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:24.199 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:24.219 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:24.219 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:24.237 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:24.237 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:24.246 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:24.246 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:24.246 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:24.246 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:24.254 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:24.255 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:24.256 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:24.257 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:24.264+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:24.273 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:24.273 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:24.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:24.291 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:24.308 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:24.309 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:24.327 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:24.327 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:24.355 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:26:24.389 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:24.387+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:24.390 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:24.390 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:24.390 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:24.402 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:24.459 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:24.466+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:24.560 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:24.567+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:24.562 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:24.562 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.843807682000005, 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:24.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:24.567 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.574114144999996, 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:24.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:24.571 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.545026378999999, 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.572 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:26:24.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:26:24.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:26:24.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.429874683000001, 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.577 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.578 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.579 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:24.580 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:24.582 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.306228261000001, 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.583 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:24.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:24.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.306157114000001, 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.588 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:26:24.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.305811384000002, 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.593 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:24.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.597 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.305770095, 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:26:24.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21117 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 413696~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.091712846, 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003161652, 2024-05-08T08:26:24.602 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21117 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:24.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477595+0000", 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477596+0000", 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8050000000000001e-06 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477599+0000", 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.616e-06 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477604+0000", 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.634e-06 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477608+0000", 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6879999999999998e-06 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480366+0000", 2024-05-08T08:26:24.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0027578020000000002 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480420+0000", 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3735e-05 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480756+0000", 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00033637199999999998 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.606 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 55.091689987999999, 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.607 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:24.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:24.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21738 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 450560~36864] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.945176830999998, 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031591710000000001, 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.611 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21738 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624131+0000", 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7300000000000001e-07 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624133+0000", 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.091e-06 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624140+0000", 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4799999999999998e-06 2024-05-08T08:26:24.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.624145+0000", 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.853e-06 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627084+0000", 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0029395129999999999 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627161+0000", 2024-05-08T08:26:24.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7096999999999997e-05 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.627290+0000", 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00012866399999999999 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.799476157999997, 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.615 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:24.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.619 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.072274122000003, 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.623 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.624 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 53.853293757000003, 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:24.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:24.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:24.629 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.365068391999998, 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:24.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.632 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.633 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.444804237, 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:24.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:24.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.296798103, 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:24.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:24.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:24.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.642 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.205937208999998, 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:24.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:24.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:26:24.646 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.824806966000001, 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:26:24.650 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:91376 3.9 e76/18 3:97e339a7:::rbd_data.1191eeab3f85.0000000000000259:head v 76'288, mlcod=76'286)", 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.51916709, 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035121610000000002, 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:24.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050141+0000", 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.95e-07 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:24.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050150+0000", 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050168+0000", 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7751e-05 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050177+0000", 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7010000000000008e-06 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050242+0000", 2024-05-08T08:26:24.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4276999999999995e-05 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050266+0000", 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4443e-05 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053607+0000", 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033412189999999999 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053653+0000", 2024-05-08T08:26:24.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5287000000000002e-05 2024-05-08T08:26:24.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:24.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:24.655 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:24.656 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:24.699 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:24.707+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:24.761 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:24.769+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:24.767 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:24.767 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:24.767 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:24.767 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:24.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:24.863 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:24.870+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:24.926 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:24.926 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:24.926 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:24.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:24.964 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:24.971+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893610848000002, 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893600763000002, 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893590283999998, 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893573357999998, 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:25.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:26:25.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:26:25.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893552438, 2024-05-08T08:26:25.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:26:25.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:25.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893514123000003, 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893490376999999, 2024-05-08T08:26:25.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893467534000003, 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893443144999999, 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893421224999997, 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893399991000003, 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.089 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.091 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.893378546999998, 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.093 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.097 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.892394242999998, 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:25.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:26:25.101 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.892373042000003, 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.102 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.892361571999999, 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:26:25.106 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:26:25.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.110 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.892351812000001, 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.111 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.892342022999998, 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:26:25.115 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:26:25.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:26:25.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.892331499000001, 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.120 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:25.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.892321819000003, 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:25.124 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:26:25.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.6690606700000004, 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:26:25.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:26:25.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:25.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:25.134 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:25.135 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:25.136 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:25.143+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:25.236 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:25.244+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:25.246 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:25.246 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:25.246 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:25.246 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:25.255 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:25.338 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:25.345+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:25.359 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:25.360 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:25.360 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:25.360 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:25.360 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:25.368 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:25.376 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:26:25.438 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:25.445+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:25.488 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:32844 3.1a 3:5c5225a1:::rbd_object_map.11915c46bf0c:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 38.338065864000001, 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0015158669999999, 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 32844 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157073+0000", 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157079+0000", 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157093+0000", 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4107999999999999e-05 2024-05-08T08:26:25.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157102+0000", 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0049999999999993e-06 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157167+0000", 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4935000000000001e-05 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157178+0000", 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158227+0000", 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.001049232 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158355+0000", 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012803000000000001 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158587+0000", 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023214899999999999 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91168 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 595968~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.648594748000001, 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040256068999999998, 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91168 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846543+0000", 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:26:25.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846545+0000", 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3269999999999999e-06 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846552+0000", 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0500000000000003e-06 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846557+0000", 2024-05-08T08:26:25.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886639+0000", 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040081952999999997 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886711+0000", 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2200000000000007e-05 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886798+0000", 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6879000000000006e-05 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91169 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 624640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.648573316, 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040335524999999997, 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.498 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91169 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846755+0000", 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000191733 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846759+0000", 2024-05-08T08:26:25.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3950000000000001e-06 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846769+0000", 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6870000000000006e-06 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846775+0000", 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4189999999999999e-06 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886812+0000", 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040036869000000003 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886850+0000", 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8130999999999998e-05 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886899+0000", 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9291e-05 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.502 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91170 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 653312~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.648352192000001, 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040303487999999998, 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91170 2024-05-08T08:26:25.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:25.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846788+0000", 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8600000000000001e-06 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846795+0000", 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2659999999999998e-06 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846800+0000", 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886930+0000", 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040129801999999999 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887002+0000", 2024-05-08T08:26:25.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2261999999999994e-05 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887088+0000", 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6491999999999998e-05 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.507 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91171 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 681984~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.648330529000001, 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040378757000000001, 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91171 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847633+0000", 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00082650900000000001 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847638+0000", 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7190000000000001e-06 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847652+0000", 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3886999999999999e-05 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847659+0000", 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5480000000000004e-06 2024-05-08T08:26:25.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887101+0000", 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.03944164 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887135+0000", 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4434000000000003e-05 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887185+0000", 2024-05-08T08:26:25.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0019999999999999e-05 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91176 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 825344~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.647377126, 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040222373999999998, 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91176 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.89e-07 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847763+0000", 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847771+0000", 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2190000000000004e-06 2024-05-08T08:26:25.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847775+0000", 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299999999999996e-06 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887825+0000", 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040049117000000002 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887897+0000", 2024-05-08T08:26:25.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2799000000000006e-05 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887982+0000", 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4901999999999995e-05 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91177 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 854016~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.647355609, 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040298390000000003, 2024-05-08T08:26:25.516 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91177 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847782+0000", 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.53e-07 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847784+0000", 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847791+0000", 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9e-06 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847795+0000", 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5460000000000002e-06 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887995+0000", 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040199657999999999 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888031+0000", 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6397999999999999e-05 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888080+0000", 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8568e-05 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91178 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 882688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:25.520 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.647335139999999, 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040467768000000001, 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91178 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000001e-07 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847804+0000", 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847810+0000", 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3510000000000001e-06 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847815+0000", 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888111+0000", 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040296697999999999 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888183+0000", 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2078999999999999e-05 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888270+0000", 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6248999999999994e-05 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91179 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 911360~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.647317087999999, 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040560845999999998, 2024-05-08T08:26:25.524 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91179 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847822+0000", 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8080000000000001e-06 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847828+0000", 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6180000000000003e-06 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847840+0000", 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2313999999999999e-05 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888294+0000", 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040453917999999998 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888332+0000", 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7960999999999998e-05 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888381+0000", 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8807000000000002e-05 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91180 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 940032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.647290333000001, 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040721286000000002, 2024-05-08T08:26:25.528 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91180 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849630+0000", 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017837389999999999 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849635+0000", 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1330000000000001e-06 2024-05-08T08:26:25.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849644+0000", 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6239999999999997e-06 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849651+0000", 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6250000000000001e-06 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888411+0000", 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.038760599 2024-05-08T08:26:25.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888483+0000", 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1863e-05 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888568+0000", 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4703000000000005e-05 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91191 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1255424~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.532 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645234050999999, 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040259947999999997, 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91191 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:25.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849906+0000", 2024-05-08T08:26:25.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849913+0000", 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6979999999999998e-06 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849920+0000", 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3350000000000003e-06 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890079+0000", 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040159261000000002 2024-05-08T08:26:25.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890115+0000", 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5876e-05 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890163+0000", 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8211999999999999e-05 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91192 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1284096~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645211818, 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040427880999999999, 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91192 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849926+0000", 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849928+0000", 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849933+0000", 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3519999999999997e-06 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849938+0000", 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1780000000000002e-06 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890194+0000", 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040255304999999998 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890266+0000", 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1979999999999999e-05 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890353+0000", 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7606000000000001e-05 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.540 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91194 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1312768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.64519265, 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040505864000000003, 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91194 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849945+0000", 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849947+0000", 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1490000000000001e-06 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849953+0000", 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849958+0000", 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890365+0000", 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040407605999999999 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890401+0000", 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6044000000000003e-05 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890450+0000", 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8983000000000002e-05 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.544 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91195 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1341440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645174101, 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040679119999999999, 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91195 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:25.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849966+0000", 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.345e-06 2024-05-08T08:26:25.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849972+0000", 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3550000000000001e-06 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849977+0000", 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1510000000000003e-06 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890481+0000", 2024-05-08T08:26:25.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040503458999999999 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890551+0000", 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0686999999999996e-05 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890642+0000", 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0799999999999998e-05 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.548 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91196 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1370112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645154278, 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040756042999999999, 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91196 2024-05-08T08:26:25.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:25.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849985+0000", 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0619999999999999e-06 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849992+0000", 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1389999999999998e-06 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849998+0000", 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890655+0000", 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040657390000000002 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890689+0000", 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4453999999999999e-05 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890739+0000", 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9363999999999997e-05 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91197 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1398784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645133791999999, 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040926395999999997, 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91197 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850004+0000", 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850006+0000", 2024-05-08T08:26:25.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.978e-06 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850012+0000", 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6680000000000004e-06 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850017+0000", 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5879999999999999e-06 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890769+0000", 2024-05-08T08:26:25.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040752661000000003 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890842+0000", 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2626999999999994e-05 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890930+0000", 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7548999999999998e-05 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91198 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1427456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.556 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645098767, 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041001199000000002, 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91198 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850039+0000", 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2900000000000004e-07 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850041+0000", 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.261e-06 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850049+0000", 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8059999999999995e-06 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850059+0000", 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8039999999999996e-06 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890955+0000", 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040895920000000002 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890990+0000", 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5760999999999999e-05 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891039+0000", 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9118e-05 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91199 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1456128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645072119, 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041163687999999997, 2024-05-08T08:26:25.560 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91199 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.39e-07 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850073+0000", 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1500000000000002e-06 2024-05-08T08:26:25.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850084+0000", 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1827e-05 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850090+0000", 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6110000000000001e-06 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891070+0000", 2024-05-08T08:26:25.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040979801000000003 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891141+0000", 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1193999999999996e-05 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891229+0000", 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7665999999999997e-05 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91200 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1484800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.564 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.645040570999999, 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041243876999999998, 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91200 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851628+0000", 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001531824 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851632+0000", 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8369999999999999e-06 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851642+0000", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4979999999999995e-06 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851651+0000", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1859999999999998e-06 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891254+0000", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.039602740999999997 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891292+0000", 2024-05-08T08:26:25.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8000999999999998e-05 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891340+0000", 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8789999999999999e-05 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91208 3.1c 3:3a43cbd7:::rbd_data.1191eeab3f85.000000000000003c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 43008] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 14.643386373, 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04166094, 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.568 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91208 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1799999999999995e-07 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851754+0000", 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3640000000000002e-06 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851759+0000", 2024-05-08T08:26:25.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5389999999999998e-06 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851764+0000", 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.544e-06 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891864+0000", 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040100363 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892007+0000", 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000142826 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892222+0000", 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021523000000000001 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893238+0000", 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001016018 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893322+0000", 2024-05-08T08:26:25.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4320000000000006e-05 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893376+0000", 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3393000000000002e-05 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893412+0000", 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5824999999999997e-05 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.573 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.574 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.574 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:25.575 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:25.575 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:25.582+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:25.675 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:25.682+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:25.679 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:25.679 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:25.679 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:25.679 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:25.688 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:25.776 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:25.783+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:25.835 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:25.835 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:25.835 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:25.835 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:25.835 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:25.844 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:25.888 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:25.895+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 56.297770886999999, 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.952 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:25.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:25.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.044289866, 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.957 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:25.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:25.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.961 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.044237918, 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:25.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:25.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.044203062999998, 2024-05-08T08:26:25.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:25.969 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.044172799000002, 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.042863433000001, 2024-05-08T08:26:25.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.042842671000002, 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:25.978 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:25.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.042771201000001, 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:25.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:25.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:25.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.986 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.042743833000003, 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:25.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:25.990 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.040789443000001, 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:25.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.040769875999999, 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:25.994 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:25.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:25.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:25.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:25.998 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.040750105999997, 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:25.999 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.000 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:26.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.040731108000003, 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:26.003 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:26.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.038831471999998, 2024-05-08T08:26:26.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:26.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.009 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:26.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:26.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.038811504999998, 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.012 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.038789932, 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.016 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:26.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:26.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:26.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.038665942000002, 2024-05-08T08:26:26.020 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:26.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:26.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.024 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.038637147000003, 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.025 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:26.028 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.036719609999999, 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.029 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:26.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:26.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 34.036698156, 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:26.033 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:26.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.037 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.038 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.038 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:26.039 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:26.039 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:26.046+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:26.140 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:26.146+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:26.187 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:26.187 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:26.187 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:26.187 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:26.195 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:26.279 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:26.286+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:26.339 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:26.341 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:26.341 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:26.341 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:26.341 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:26.341 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:26.342 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:26.350+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:26.350 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:26.442 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:26.449+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:26.502 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:26.503 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.73340288, 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.507 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.511 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733382162000002, 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733354224999999, 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.516 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:26.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733332462, 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.520 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:26.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:26.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:26.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733309884000001, 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:26.524 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:26.528 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733269815, 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:26.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:26.532 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733247507000002, 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733218925999999, 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:26.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733194673, 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:26.541 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.733162275000002, 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:26.545 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.548 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.731824469999999, 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:26.549 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.731802452, 2024-05-08T08:26:26.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:26.557 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.731760092999998, 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:26.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:26.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.731718573999999, 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.562 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:26.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.729740478, 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.566 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:26.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:26.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.729729184, 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.570 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.725641407000001, 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.574 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:26.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:26.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:26.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.72562593, 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.578 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:26.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:26.580 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.582 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.720507155, 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:26.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:26.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:26.586 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 24.720229226000001, 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:26.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:26.590 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:26.591 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:26.591 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:26.592 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:26.592 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:26.593 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:26.593 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:26.599+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:26.593 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:26.593 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:26.594 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:26.594 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:26.594 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:26.594 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:26.611 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:26.611 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:26.629 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:26.630 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:26.692 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:26.700+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:26.739 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:26.739 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:26.748 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:26.792 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:26.800+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:26.853 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:26.853 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:26.862 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:26.894 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:26.901+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:26.951 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:26.951 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:26.951 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T08:26:26.952 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.000000 2024-05-08T08:26:26.952 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 0 2024-05-08T08:26:26.968 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:26.968 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:26.976 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:26.995 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:27.003+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:27.095 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:27.102+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:27.107 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:27.107 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:27.116 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:27.196 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:27.204+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:27.266 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:27.266 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:27.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:27.320 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:27.327+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:27.386 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 0 (802) 2024-05-08T08:26:27.389 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:27.389 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:27.396 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:26:27.398 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:27.405+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:27.400 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:27.498 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:27.506+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:27.554 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:27.554 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:27.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:27.599 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:27.607+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:27.672 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:27.673 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:27.682 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:27.726 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:27.734+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:27.792 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:27.792 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:27.801 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:27.801 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:27.808+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:27.901 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:27.908+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:27.951 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:27.952 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:27.960 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:28.001 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:28.009+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:28.102 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:28.109+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:28.111 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:28.111 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:28.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:28.203 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:28.211+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:28.269 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:28.270 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:28.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:28.320 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:28.327+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:28.389 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:26:28.404 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:28.411+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:28.436 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:28.436 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:28.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:28.505 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:28.511+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:28.598 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:28.598 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:28.606 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:28.613+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:28.607 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:28.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:28.707 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:28.714+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:28.759 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:28.759 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:28.769 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:28.774 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:28.775 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:28.792 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:28.792 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:28.808 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:28.815+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:28.809 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:28.810 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:28.826 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:28.827 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:28.844 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:28.844 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:28.863 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:28.864 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:28.878 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:28.878 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:28.878 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:28.881 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:28.881 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:28.888 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:28.898 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:28.899 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:28.909 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:28.916+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:28.998 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:28.998 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:28.998 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:28.998 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:28.998 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:29.007 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:29.010 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:29.017+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:29.111 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:29.118+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:32844 3.1a 3:5c5225a1:::rbd_object_map.11915c46bf0c:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 41.972449920000003, 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0015158669999999, 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.123 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 32844 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157073+0000", 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157079+0000", 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157093+0000", 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4107999999999999e-05 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157102+0000", 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0049999999999993e-06 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157167+0000", 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4935000000000001e-05 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157178+0000", 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158227+0000", 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.001049232 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158355+0000", 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012803000000000001 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158587+0000", 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023214899999999999 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.127 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91168 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 595968~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.282978803999999, 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040256068999999998, 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91168 2024-05-08T08:26:29.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846543+0000", 2024-05-08T08:26:29.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846545+0000", 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3269999999999999e-06 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846552+0000", 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0500000000000003e-06 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846557+0000", 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886639+0000", 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040081952999999997 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886711+0000", 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2200000000000007e-05 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886798+0000", 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6879000000000006e-05 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91169 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 624640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.282957371999998, 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040335524999999997, 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.132 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91169 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:29.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846755+0000", 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000191733 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846759+0000", 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3950000000000001e-06 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846769+0000", 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6870000000000006e-06 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846775+0000", 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4189999999999999e-06 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886812+0000", 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040036869000000003 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886850+0000", 2024-05-08T08:26:29.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8130999999999998e-05 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886899+0000", 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9291e-05 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91170 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 653312~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.282736247999999, 2024-05-08T08:26:29.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040303487999999998, 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91170 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846788+0000", 2024-05-08T08:26:29.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8600000000000001e-06 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846795+0000", 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2659999999999998e-06 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846800+0000", 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886930+0000", 2024-05-08T08:26:29.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040129801999999999 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887002+0000", 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2261999999999994e-05 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887088+0000", 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6491999999999998e-05 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91171 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 681984~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.282714585000001, 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040378757000000001, 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91171 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847633+0000", 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00082650900000000001 2024-05-08T08:26:29.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847638+0000", 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7190000000000001e-06 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847652+0000", 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3886999999999999e-05 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847659+0000", 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5480000000000004e-06 2024-05-08T08:26:29.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887101+0000", 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.03944164 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887135+0000", 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4434000000000003e-05 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887185+0000", 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0019999999999999e-05 2024-05-08T08:26:29.144 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91176 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 825344~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.281761182, 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040222373999999998, 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91176 2024-05-08T08:26:29.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:29.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.89e-07 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847763+0000", 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847771+0000", 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2190000000000004e-06 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847775+0000", 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299999999999996e-06 2024-05-08T08:26:29.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887825+0000", 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040049117000000002 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887897+0000", 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2799000000000006e-05 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887982+0000", 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4901999999999995e-05 2024-05-08T08:26:29.148 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91177 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 854016~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.281739665, 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040298390000000003, 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.149 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91177 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847782+0000", 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.53e-07 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847784+0000", 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847791+0000", 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9e-06 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847795+0000", 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5460000000000002e-06 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887995+0000", 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040199657999999999 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888031+0000", 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6397999999999999e-05 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888080+0000", 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8568e-05 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91178 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 882688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.281719196000001, 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040467768000000001, 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.153 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91178 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000001e-07 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847804+0000", 2024-05-08T08:26:29.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847810+0000", 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3510000000000001e-06 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847815+0000", 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888111+0000", 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040296697999999999 2024-05-08T08:26:29.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888183+0000", 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2078999999999999e-05 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888270+0000", 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6248999999999994e-05 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.157 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91179 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 911360~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.281701143999999, 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040560845999999998, 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91179 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.159 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847822+0000", 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8080000000000001e-06 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847828+0000", 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6180000000000003e-06 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847840+0000", 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2313999999999999e-05 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888294+0000", 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040453917999999998 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888332+0000", 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7960999999999998e-05 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888381+0000", 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8807000000000002e-05 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91180 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 940032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.281674388999999, 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040721286000000002, 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91180 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.162 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849630+0000", 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017837389999999999 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.163 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849635+0000", 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1330000000000001e-06 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849644+0000", 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6239999999999997e-06 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849651+0000", 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6250000000000001e-06 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.164 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888411+0000", 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.038760599 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888483+0000", 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1863e-05 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888568+0000", 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4703000000000005e-05 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91191 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1255424~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279618107000001, 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040259947999999997, 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91191 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.166 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:29.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849906+0000", 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849913+0000", 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6979999999999998e-06 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849920+0000", 2024-05-08T08:26:29.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3350000000000003e-06 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890079+0000", 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040159261000000002 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890115+0000", 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5876e-05 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890163+0000", 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8211999999999999e-05 2024-05-08T08:26:29.169 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91192 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1284096~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279595874000002, 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040427880999999999, 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.170 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91192 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849926+0000", 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849928+0000", 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849933+0000", 2024-05-08T08:26:29.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3519999999999997e-06 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849938+0000", 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1780000000000002e-06 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890194+0000", 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040255304999999998 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890266+0000", 2024-05-08T08:26:29.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1979999999999999e-05 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890353+0000", 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7606000000000001e-05 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91194 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1312768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279576706, 2024-05-08T08:26:29.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040505864000000003, 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91194 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849945+0000", 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.176 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849947+0000", 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1490000000000001e-06 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849953+0000", 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849958+0000", 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890365+0000", 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040407605999999999 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890401+0000", 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6044000000000003e-05 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890450+0000", 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8983000000000002e-05 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.178 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91195 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1341440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279558157, 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040679119999999999, 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91195 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.179 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:29.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849966+0000", 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.345e-06 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849972+0000", 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3550000000000001e-06 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849977+0000", 2024-05-08T08:26:29.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1510000000000003e-06 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890481+0000", 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040503458999999999 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890551+0000", 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0686999999999996e-05 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890642+0000", 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0799999999999998e-05 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91196 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1370112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279538334000001, 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040756042999999999, 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.183 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91196 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849985+0000", 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0619999999999999e-06 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849992+0000", 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1389999999999998e-06 2024-05-08T08:26:29.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849998+0000", 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890655+0000", 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040657390000000002 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890689+0000", 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4453999999999999e-05 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890739+0000", 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9363999999999997e-05 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91197 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1398784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279517848000001, 2024-05-08T08:26:29.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040926395999999997, 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91197 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850004+0000", 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850006+0000", 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.978e-06 2024-05-08T08:26:29.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850012+0000", 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6680000000000004e-06 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850017+0000", 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5879999999999999e-06 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890769+0000", 2024-05-08T08:26:29.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040752661000000003 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890842+0000", 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2626999999999994e-05 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890930+0000", 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7548999999999998e-05 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91198 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1427456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279482822999999, 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041001199000000002, 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91198 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.192 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850039+0000", 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2900000000000004e-07 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850041+0000", 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.261e-06 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850049+0000", 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8059999999999995e-06 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850059+0000", 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8039999999999996e-06 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890955+0000", 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040895920000000002 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890990+0000", 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5760999999999999e-05 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891039+0000", 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9118e-05 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.195 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91199 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1456128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279456175, 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041163687999999997, 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91199 2024-05-08T08:26:29.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.39e-07 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850073+0000", 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1500000000000002e-06 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850084+0000", 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1827e-05 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850090+0000", 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6110000000000001e-06 2024-05-08T08:26:29.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891070+0000", 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040979801000000003 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891141+0000", 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1193999999999996e-05 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891229+0000", 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7665999999999997e-05 2024-05-08T08:26:29.199 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91200 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1484800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.279424627000001, 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041243876999999998, 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91200 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.200 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851628+0000", 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001531824 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851632+0000", 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8369999999999999e-06 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851642+0000", 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4979999999999995e-06 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851651+0000", 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1859999999999998e-06 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891254+0000", 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.039602740999999997 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891292+0000", 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8000999999999998e-05 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891340+0000", 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8789999999999999e-05 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91208 3.1c 3:3a43cbd7:::rbd_data.1191eeab3f85.000000000000003c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 43008] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 18.277770429, 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04166094, 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.204 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91208 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1799999999999995e-07 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851754+0000", 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3640000000000002e-06 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851759+0000", 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5389999999999998e-06 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851764+0000", 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.544e-06 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891864+0000", 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040100363 2024-05-08T08:26:29.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892007+0000", 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000142826 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892222+0000", 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021523000000000001 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893238+0000", 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001016018 2024-05-08T08:26:29.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893322+0000", 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4320000000000006e-05 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893376+0000", 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3393000000000002e-05 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893412+0000", 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5824999999999997e-05 2024-05-08T08:26:29.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.210 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.210 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.210 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.210 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.210 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:29.211 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:29.253 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:29.260+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:29.313 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:29.320+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:29.358 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:29.358 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:29.359 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:29.367 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:29.451 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:29.459+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:29.512 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:29.512 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:29.512 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:29.512 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:29.512 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:29.514 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:29.522+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:29.521 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:29.615 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:29.623+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:29.667 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 60.013615373999997, 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.668 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.669 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:29.670 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.671 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.672 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.673 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.760134352999998, 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:29.674 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.675 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.676 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.677 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.760082404999999, 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.678 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.679 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:29.680 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.681 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.682 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.760047550000003, 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.683 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.684 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.685 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.686 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.760017286, 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:29.687 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.688 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:29.689 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.690 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.758707919999999, 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.691 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.692 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.693 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:29.694 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.758687158000001, 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.695 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.696 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.697 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.758615687999999, 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.699 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:29.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:29.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:29.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.758588320000001, 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.703 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:29.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:29.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.75663393, 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.707 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:29.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:29.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.711 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.756614362999997, 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:29.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.715 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.756594593000003, 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.756575595000001, 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:29.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:29.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:29.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.754675958999997, 2024-05-08T08:26:29.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.728 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.754655991999996, 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.729 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:29.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.754634418999999, 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.733 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.737 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.754510429, 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:29.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:29.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:29.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.754481634000001, 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.752564096999997, 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.749 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 37.752542642999998, 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:29.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.753 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.754 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.754 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:29.754 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:29.754 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:29.755 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:29.755 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:29.762+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:29.856 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:29.863+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:29.901 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:29.902 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:29.902 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:29.911 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:29.995 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:30.003+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:30.059 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:30.066+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:30.069 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:30.069 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:30.069 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:30.070 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:30.070 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:30.079 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:30.163 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:30.171+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:30.226 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:30.226 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:30.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452967845, 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.227 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:30.230 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452947127000002, 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:30.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:30.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.234 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452919189999999, 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.235 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452897427, 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:30.242 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452874849000001, 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.45283478, 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.247 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:30.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452812472000002, 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.251 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452783890999999, 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.255 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:30.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.259 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452759638, 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:30.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:30.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:30.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.452727240000002, 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.264 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:30.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.268 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.451389434999999, 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.451367417, 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:30.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.276 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.451325057999998, 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.280 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.451283538999999, 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:30.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:30.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:30.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.449305443, 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.449294149, 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.290 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.292 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.445206372000001, 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:30.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:30.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.445190895, 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:30.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.44007212, 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:30.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:30.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.304 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.439794191000001, 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.305 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.309 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:30.310 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:30.311 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:30.318+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:30.408 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:26:30.411 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:30.418+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:30.457 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:30.457 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:30.457 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:30.466 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:30.511 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:30.519+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:30.572 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:30.572 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:30.572 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:30.572 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:30.572 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:30.581 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:30.622 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:30.630+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:30.713 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:30.720+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:30.729 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 93.170313304999993, 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:30.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:30.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:30.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:30.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:30.735 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.946207997999998, 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.946122844000001, 2024-05-08T08:26:30.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:30.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.743 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.946094938000002, 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:30.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:30.747 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.946066537, 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.748 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:30.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:30.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:30.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945982397999998, 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:30.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:30.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:30.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:30.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945901902999999, 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.756 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945869323, 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.760 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:30.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.764 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945816132000001, 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:30.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:30.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:30.768 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945790153000001, 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:30.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:30.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.772 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945734820999999, 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945584788000001, 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:30.776 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:30.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:30.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:30.780 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945535129, 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:30.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:30.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.784 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945500559999999, 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:30.788 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945478553000001, 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:30.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:30.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.792 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945455579000001, 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.945425901, 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.797 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:30.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:30.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.944203323, 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.801 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:30.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.804 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.944192828999999, 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.805 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.808 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 28.939073993000001, 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:30.809 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:30.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.812 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:30.813 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:30.813 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:30.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:30.814 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:30.821+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:30.911 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:30.924 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:30.925 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:30.925 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:30.925 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:30.925 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:30.932+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:30.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:30.970 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:26:31.015 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:31.022+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:31.058 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:31.058 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:31.077 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:31.078 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:31.096 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:31.096 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:31.102 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:31.102 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:31.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:31.114 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:31.114 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:31.115 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:31.123+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:31.131 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:31.132 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:31.149 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:31.149 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:31.167 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:31.167 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:31.185 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:31.185 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:31.222 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:31.222 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:31.222 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:31.229+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:31.231 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:31.316 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:31.323+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:31.381 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:31.381 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:31.390 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:31.431 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:31.439+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:31.519 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:31.526+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:31.540 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:31.540 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:31.549 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:31.619 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:31.627+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:31.658 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:31.659 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:31.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:31.720 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:31.728+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:31.777 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:31.777 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:31.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:31.833 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:31.841+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:31.899 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:31.908 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:31.949 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:31.957+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:32.022 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:32.030+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:32.068 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:32.068 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:32.079 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:32.122 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:32.130+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:32.223 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:32.231+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:32.229 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:32.229 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:32.238 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:32.324 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:32.332+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:32.388 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:32.388 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:32.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:32.403 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:32.404 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:32.404 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T08:26:32.404 INFO:tasks.thrashosds.thrasher:Setting osd 2 primary_affinity to 0.580231 2024-05-08T08:26:32.404 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 0.580230566887461 2024-05-08T08:26:32.425 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:32.433+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:32.526 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:32.533+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:32.549 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:32.549 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:32.559 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:32.627 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:32.635+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:32.666 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:32.666 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:32.675 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:32.759 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:32.767+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:32.821 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:32.821 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:32.829 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:32.836+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:32.830 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:32.929 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:32.937+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:32.977 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:32.977 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:32.984 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 0.580231 (8380252) 2024-05-08T08:26:32.987 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:32.994 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:26:33.031 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:33.039+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:33.100 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:33.101 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:33.110 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:33.151 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:33.159+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:33.197 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:33.240 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:33.248+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:33.255 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:33.255 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:33.264 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:33.325 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:33.325 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:33.332 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:33.340+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:33.342 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:33.342 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:33.359 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:33.359 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:33.369 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:33.369 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:33.376 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:33.376 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:33.379 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:33.394 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:33.394 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:33.412 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:33.412 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:33.429 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:33.429 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:33.434 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:33.441+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:33.447 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:33.447 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:33.486 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:33.486 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:33.486 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:33.486 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:33.495 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:33.536 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:33.544+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:33.636 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:33.643+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:33.640 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:33.640 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:33.640 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:33.648 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:33.736 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:33.744+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:33.756 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:33.756 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:33.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:33.756 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:33.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.756 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.756 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982532763000002, 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:33.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:33.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982512045, 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.761 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:33.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:33.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982484108000001, 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:33.765 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.769 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982462344999998, 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.770 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.773 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982439766999999, 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:33.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:33.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:33.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982399698000002, 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.778 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:33.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:33.782 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.98237739, 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.783 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.784 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.785 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.786 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982348809000001, 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:33.787 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.788 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.789 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.790 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982324556000002, 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.791 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:33.792 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:33.793 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:33.794 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.982292158, 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.795 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.796 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.797 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.798 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:33.799 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.980954353000001, 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:33.800 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:33.801 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.802 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.803 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.980932334999999, 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.804 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.805 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:33.806 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:33.807 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.980889976, 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.808 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.809 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.810 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.811 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.980848457, 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.812 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.813 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.814 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.815 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:33.816 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.978870360999998, 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:33.817 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.818 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.819 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:33.820 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.978859066999998, 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.821 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.822 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.823 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.824 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.97477129, 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.825 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.828 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.829 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.974755813000002, 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.830 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:33.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.833 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.833 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.833 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:33.833 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:33.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.833 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:33.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.969637037999998, 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.836 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:33.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 31.969359108999999, 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:33.840 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.841 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:33.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:33.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:33.844 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:33.846 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:33.846 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:33.853+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:33.946 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:33.954+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:33.966 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:26:33.997 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:33.997 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:33.997 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:33.997 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:34.006 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:34.049 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:34.057+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:34.123 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:34.123 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:34.123 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:34.123 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:34.123 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:34.132 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:34.173 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:34.181+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:34.249 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:34.257+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:34.279 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:34.279 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 96.720276006999995, 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:34.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:34.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.4961707, 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:34.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.289 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:34.290 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.496085546000003, 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.291 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.292 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.293 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.294 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.496057639999997, 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:34.295 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:34.296 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:34.297 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.298 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.496029239000002, 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.299 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.300 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.301 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:34.302 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:34.303 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.4959451, 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:34.304 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:34.305 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:34.306 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:34.307 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.308 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495864605000001, 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.309 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:34.310 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.311 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.312 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:34.313 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495832024999999, 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.314 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.315 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.316 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.317 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495778833999999, 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.318 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:34.319 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.320 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.321 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.322 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495752854999999, 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:34.323 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.324 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.325 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.326 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495697522999997, 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:34.327 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.328 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:34.329 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.330 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.331 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.49554749, 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.332 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.333 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.334 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:34.335 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495497831000002, 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.336 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.337 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.338 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.339 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495463262000001, 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.340 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.341 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.342 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:34.343 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:34.344 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495441255000003, 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.345 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.346 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.347 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.348 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:34.349 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495418280999999, 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.350 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:34.351 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:34.352 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:34.353 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.495388603000002, 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.354 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.494166024999998, 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.358 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:34.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.494155530999997, 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.489036695000003, 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:34.367 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:34.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:34.371 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:34.373 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:34.374 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:34.380+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:34.473 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:34.481+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:34.489 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:34.489 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:34.489 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:34.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:34.575 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:34.582+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:34.608 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:34.608 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:34.608 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:34.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:34.675 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:34.683+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:34.769 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:34.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.615547743, 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:34.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:26:34.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:26:34.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.615526643000003, 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 42.615505984999999, 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108422 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781502108, 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032000842000000002, 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108422 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994307+0000", 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994314+0000", 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-05-08T08:26:34.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994320+0000", 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7760000000000003e-06 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026109+0000", 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031789339 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026194+0000", 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4841999999999998e-05 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026301+0000", 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106667 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108428 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781474499, 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032197602999999998, 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.786 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108428 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994328+0000", 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994330+0000", 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5689999999999999e-06 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994338+0000", 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5830000000000001e-06 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994343+0000", 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026333+0000", 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031990022 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026419+0000", 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6278000000000003e-05 2024-05-08T08:26:34.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026525+0000", 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010569 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108429 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781451219000001, 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032394017999999997, 2024-05-08T08:26:34.790 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108429 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994353+0000", 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3829999999999999e-06 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994360+0000", 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6850000000000004e-06 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994365+0000", 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2680000000000002e-06 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026559+0000", 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032193927999999997 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026647+0000", 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7882999999999999e-05 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026745+0000", 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7460999999999994e-05 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108430 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3211264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.794 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781429251, 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032603935000000001, 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108430 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994375+0000", 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3989999999999998e-06 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994383+0000", 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2869999999999999e-06 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994387+0000", 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026777+0000", 2024-05-08T08:26:34.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032389793 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026865+0000", 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7763999999999995e-05 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026976+0000", 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111488 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108431 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3244032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781406843999999, 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032796189000000003, 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108431 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.799 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:26:34.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994398+0000", 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994404+0000", 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3969999999999999e-06 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994410+0000", 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027010+0000", 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032600259999999999 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027094+0000", 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128999999999994e-05 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027191+0000", 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7089000000000005e-05 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.802 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108444 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781384974, 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032973852999999997, 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108444 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994419+0000", 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2900000000000001e-06 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994426+0000", 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5080000000000002e-06 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994431+0000", 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0200000000000002e-06 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027223+0000", 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032791703999999998 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027303+0000", 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0068999999999995e-05 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027391+0000", 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7899000000000007e-05 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108445 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781363949999999, 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033150011, 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108445 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994441+0000", 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994448+0000", 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9929999999999998e-06 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994452+0000", 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8199999999999996e-06 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.809 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027420+0000", 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032967989000000003 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027500+0000", 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9332000000000005e-05 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027588+0000", 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8080999999999999e-05 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108446 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781341691, 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033416003, 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108446 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994461+0000", 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994463+0000", 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994470+0000", 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4489999999999996e-06 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994475+0000", 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027699+0000", 2024-05-08T08:26:34.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033223821000000001 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027779+0000", 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0411e-05 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027876+0000", 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6855999999999996e-05 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.814 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108447 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.781319647, 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033598876, 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108447 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994483+0000", 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994485+0000", 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994492+0000", 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994497+0000", 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3569999999999997e-06 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027912+0000", 2024-05-08T08:26:34.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033415331999999999 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027991+0000", 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8825999999999994e-05 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.028081+0000", 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9858000000000006e-05 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108675 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.776323727999999, 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032085805000000002, 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108675 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:26:34.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999480+0000", 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.793e-06 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999486+0000", 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999490+0000", 2024-05-08T08:26:34.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.464e-06 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031407+0000", 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031916959000000002 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031488+0000", 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0567000000000003e-05 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031564+0000", 2024-05-08T08:26:34.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6114999999999999e-05 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108676 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.776306229999999, 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032251646000000002, 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.823 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108676 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999498+0000", 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999503+0000", 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999509+0000", 2024-05-08T08:26:34.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8270000000000003e-06 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031594+0000", 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032084797999999998 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031672+0000", 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8765999999999998e-05 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031747+0000", 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4881000000000004e-05 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108677 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.776287474, 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032405850999999999, 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.827 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108677 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999515+0000", 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.79e-07 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999516+0000", 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999522+0000", 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2460000000000003e-06 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999526+0000", 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.673e-06 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031777+0000", 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032250681000000003 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031851+0000", 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3694000000000003e-05 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031920+0000", 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9449999999999994e-05 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108678 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.776270169, 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032573421999999998, 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108678 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0800000000000004e-07 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999538+0000", 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999545+0000", 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999549+0000", 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031947+0000", 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032397903999999998 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032029+0000", 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1985999999999997e-05 2024-05-08T08:26:34.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032105+0000", 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6032999999999995e-05 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108703 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6832128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.774942825, 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03217747, 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108703 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:34.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000861+0000", 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8199999999999999e-06 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000866+0000", 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0880000000000002e-06 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000871+0000", 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510000000000004e-06 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032860+0000", 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031989640999999999 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032962+0000", 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010197200000000001 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033036+0000", 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4299000000000001e-05 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108704 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6864896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.774593721, 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032008086999999998, 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.839 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108704 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001209+0000", 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5199999999999996e-07 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001213+0000", 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8770000000000003e-06 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001224+0000", 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1342000000000001e-05 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001231+0000", 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033067+0000", 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031835544 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033149+0000", 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1719999999999997e-05 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033216+0000", 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7645999999999996e-05 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108705 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.843 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.774558365000001, 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032150629999999999, 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108705 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001244+0000", 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001247+0000", 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001254+0000", 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7190000000000001e-06 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001260+0000", 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5099999999999998e-06 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033245+0000", 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031984929000000002 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033322+0000", 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7435999999999995e-05 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033394+0000", 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1740000000000001e-05 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.847 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108706 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6930432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 14.774530672000001, 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032385262999999997, 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108706 2024-05-08T08:26:34.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001272+0000", 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:26:34.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001274+0000", 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001281+0000", 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0840000000000003e-06 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001287+0000", 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033490+0000", 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032203455999999998 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033573+0000", 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2860000000000003e-05 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033656+0000", 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3380999999999993e-05 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.851 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:34.852 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:34.852 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:34.853 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:34.860+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:34.853 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:34.953 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:34.961+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:35.008 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:35.008 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:35.008 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:35.008 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:35.018 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:35.054 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:35.062+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:35.146 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:35.146 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:35.146 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:35.147 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:35.147 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:35.155 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:35.162+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:35.156 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:35.256 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:35.263+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:35.268 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.268 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69714 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453971312, 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046246284999999998, 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69714 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820801+0000", 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-05-08T08:26:35.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820803+0000", 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820809+0000", 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7410000000000003e-06 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820815+0000", 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6200000000000004e-06 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866947+0000", 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046131923999999998 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866993+0000", 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867046+0000", 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3052e-05 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.272 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69715 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3162112~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453949156999997, 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046369965999999999, 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69715 2024-05-08T08:26:35.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820823+0000", 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820825+0000", 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.655e-06 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820832+0000", 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0849999999999999e-06 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820838+0000", 2024-05-08T08:26:35.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867075+0000", 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046236823000000003 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867133+0000", 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867192+0000", 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9475999999999998e-05 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69716 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3391488~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453925656999999, 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046544169000000003, 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.277 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69716 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:35.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820848+0000", 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820855+0000", 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140000000000002e-06 2024-05-08T08:26:35.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820861+0000", 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867234+0000", 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046372732 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867299+0000", 2024-05-08T08:26:35.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5485000000000001e-05 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867390+0000", 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969000000000003e-05 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69717 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3620864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453903662000002, 2024-05-08T08:26:35.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046712535999999999, 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69717 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820870+0000", 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1320000000000001e-06 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820879+0000", 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820892+0000", 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3356000000000001e-05 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867422+0000", 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046529981999999998 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867485+0000", 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2614999999999996e-05 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867580+0000", 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5359000000000006e-05 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69718 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453870801000001, 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046863712000000002, 2024-05-08T08:26:35.285 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69718 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820904+0000", 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820911+0000", 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820917+0000", 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867609+0000", 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046691734999999998 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867681+0000", 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1931e-05 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867764+0000", 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3356000000000006e-05 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69719 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4079616~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453847089999996, 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047037906999999997, 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69719 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820925+0000", 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820927+0000", 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820934+0000", 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.764e-06 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820939+0000", 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.553e-06 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867796+0000", 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046856496999999997 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867868+0000", 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2163999999999996e-05 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867962+0000", 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4337000000000002e-05 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69720 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4308992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453824943999997, 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047192222999999998, 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69720 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-05-08T08:26:35.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820949+0000", 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820956+0000", 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820963+0000", 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867988+0000", 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047025083000000002 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868053+0000", 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4871999999999998e-05 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868139+0000", 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5626999999999999e-05 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.297 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69721 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4538368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453800543, 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047345685999999998, 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69721 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.298 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:26:35.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820974+0000", 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820982+0000", 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8370000000000003e-06 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820988+0000", 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868167+0000", 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047179062000000001 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868232+0000", 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4889999999999997e-05 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868317+0000", 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4244000000000002e-05 2024-05-08T08:26:35.301 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69722 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4767744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453775862000001, 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047510657999999997, 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.302 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69722 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821004+0000", 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821011+0000", 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6510000000000001e-06 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821019+0000", 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:26:35.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868347+0000", 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047327859 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868419+0000", 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2725999999999995e-05 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868506+0000", 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6793999999999996e-05 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.305 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69723 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4997120~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453744344999997, 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047669162000000001, 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69723 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.306 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821028+0000", 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821030+0000", 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821037+0000", 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821043+0000", 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868536+0000", 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047493103000000002 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868611+0000", 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4647999999999994e-05 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868696+0000", 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5476999999999995e-05 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69724 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5226496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453721066, 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819591000000002, 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69724 2024-05-08T08:26:35.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821051+0000", 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821053+0000", 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821060+0000", 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.097e-06 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821066+0000", 2024-05-08T08:26:35.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868722+0000", 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047656233999999999 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868795+0000", 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2621999999999997e-05 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868870+0000", 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5352999999999995e-05 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69725 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5455872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453698774000003, 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047973017999999999, 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69725 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821075+0000", 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.356e-06 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821082+0000", 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821088+0000", 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868907+0000", 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819382000000001 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868979+0000", 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1922999999999996e-05 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869046+0000", 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6724999999999996e-05 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69726 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5685248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453659350999999, 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048091651999999999, 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.318 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69726 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821113+0000", 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821115+0000", 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7889999999999999e-06 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821124+0000", 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.191e-06 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821130+0000", 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869065+0000", 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047935187999999997 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869129+0000", 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4356999999999994e-05 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869204+0000", 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399000000000004e-05 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69727 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5914624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453633244000002, 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048288931, 2024-05-08T08:26:35.322 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69727 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821139+0000", 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821141+0000", 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821148+0000", 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821153+0000", 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869256+0000", 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048102180000000001 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869343+0000", 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7828e-05 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869427+0000", 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3666999999999997e-05 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69728 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6144000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453610796, 2024-05-08T08:26:35.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048487729, 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69728 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821164+0000", 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821171+0000", 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821176+0000", 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869489+0000", 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048312213999999999 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869566+0000", 2024-05-08T08:26:35.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7657999999999994e-05 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869648+0000", 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2194000000000006e-05 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69729 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6373376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453587874, 2024-05-08T08:26:35.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048680186, 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69729 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821186+0000", 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821192+0000", 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821197+0000", 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869699+0000", 2024-05-08T08:26:35.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048501191999999999 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869782+0000", 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2973000000000007e-05 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869864+0000", 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2241000000000001e-05 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69730 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6602752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453567675999999, 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048837093999999998, 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69730 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821206+0000", 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821212+0000", 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821218+0000", 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869928+0000", 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048710003000000002 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869995+0000", 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7125000000000005e-05 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870041+0000", 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6134999999999998e-05 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.338 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69731 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6832128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453546604000003, 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048956353000000001, 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69731 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821228+0000", 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821234+0000", 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821239+0000", 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.485e-06 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870066+0000", 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048826621000000001 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870133+0000", 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870181+0000", 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8553000000000001e-05 2024-05-08T08:26:35.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69732 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7061504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.453507715000001, 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049133148000000001, 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69732 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-05-08T08:26:35.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821267+0000", 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821286+0000", 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9122000000000001e-05 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821292+0000", 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870231+0000", 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048938161000000001 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870316+0000", 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5143999999999997e-05 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870397+0000", 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1118000000000005e-05 2024-05-08T08:26:35.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69733 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7290880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 33.45347039, 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049266982000000001, 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69733 2024-05-08T08:26:35.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:35.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821302+0000", 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821304+0000", 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821311+0000", 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729999999999998e-06 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:35.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821317+0000", 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2660000000000001e-06 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870426+0000", 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049109685 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870497+0000", 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0860999999999998e-05 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:35.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870568+0000", 2024-05-08T08:26:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0984999999999999e-05 2024-05-08T08:26:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:35.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:35.351 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:35.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:35.394 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:35.401+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:35.458 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:35.459 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:35.466+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:35.504 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:35.504 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:35.504 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:35.504 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:35.513 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:35.559 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:35.567+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:35.586 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:35.586 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:35.604 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:35.604 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:35.621 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:35.621 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:35.625 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:35.625 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:35.635 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:35.639 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:35.640 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:35.658 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:35.658 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:35.660 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:35.668+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:35.676 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:35.676 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:35.694 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:35.694 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:35.712 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:35.712 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:35.746 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:35.746 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:35.755 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:35.797 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:35.804+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:35.862 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:35.869+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:35.905 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:35.905 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:35.914 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:35.963 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:35.971+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:35.988 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:26:36.064 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:36.072+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:36.079 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.079 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.089 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:36.166 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:36.173+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:36.204 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.204 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.214 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:36.266 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:36.274+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.321 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.330 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:36.367 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:36.375+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:36.468 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:36.476+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:36.478 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.479 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.487 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:36.569 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:36.577+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:36.591 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.592 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.600 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:36.669 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:36.678+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:36.705 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.705 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.713 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:36.770 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:36.778+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:36.819 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.819 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.827 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:36.871 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:36.879+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:36.933 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:36.933 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:36.942 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:36.984 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:36.991+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:37.008 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:26:37.073 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:37.081+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:37.109 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:37.109 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:37.119 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:37.174 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:37.182+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:37.266 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:37.266 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:37.275 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:37.275 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:37.282+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:37.375 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:37.383+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:37.380 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:37.380 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:37.389 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:37.476 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:37.484+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:37.494 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:37.494 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:37.503 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:37.587 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:37.595+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:37.648 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:37.648 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:37.657 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:37.678 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:37.686+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:37.724 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:37.778 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:37.786+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:37.801 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:37.801 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:37.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:37.865 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:37.865 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:37.879 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:37.887+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:37.883 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:37.883 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:37.901 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:37.901 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:37.919 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:37.919 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:37.927 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:37.927 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:37.927 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:37.936 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:37.937 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:37.937 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:37.955 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:37.955 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:37.973 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:37.973 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:37.980 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:37.988+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:37.990 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:37.991 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:38.001 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:38.001 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:38.001 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:26:38.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:26:38.053 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:38.053 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:38.053 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:38.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:38.081 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:38.089+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:38.182 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:38.190+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:38.226 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.073068050000003, 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.227 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:38.231 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.073046949999998, 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:38.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:26:38.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:26:38.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 46.073026292000002, 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:26:38.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.239 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108422 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.239022415000001, 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032000842000000002, 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.240 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108422 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994307+0000", 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994314+0000", 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994320+0000", 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7760000000000003e-06 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026109+0000", 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031789339 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026194+0000", 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4841999999999998e-05 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026301+0000", 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106667 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108428 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238994806000001, 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032197602999999998, 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.244 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108428 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994328+0000", 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994330+0000", 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5689999999999999e-06 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994338+0000", 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5830000000000001e-06 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994343+0000", 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026333+0000", 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031990022 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026419+0000", 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6278000000000003e-05 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026525+0000", 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010569 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108429 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238971526, 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032394017999999997, 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.248 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108429 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994353+0000", 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3829999999999999e-06 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994360+0000", 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6850000000000004e-06 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994365+0000", 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2680000000000002e-06 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026559+0000", 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032193927999999997 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026647+0000", 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7882999999999999e-05 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026745+0000", 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7460999999999994e-05 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.252 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108430 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3211264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238949558000002, 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032603935000000001, 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108430 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994375+0000", 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3989999999999998e-06 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994383+0000", 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2869999999999999e-06 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994387+0000", 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026777+0000", 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032389793 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026865+0000", 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7763999999999995e-05 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026976+0000", 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111488 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108431 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3244032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238927150999999, 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032796189000000003, 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108431 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:38.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994398+0000", 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994404+0000", 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3969999999999999e-06 2024-05-08T08:26:38.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994410+0000", 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027010+0000", 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032600259999999999 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027094+0000", 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128999999999994e-05 2024-05-08T08:26:38.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027191+0000", 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7089000000000005e-05 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108444 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:38.261 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238905281000001, 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032973852999999997, 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108444 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994419+0000", 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2900000000000001e-06 2024-05-08T08:26:38.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994426+0000", 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5080000000000002e-06 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994431+0000", 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0200000000000002e-06 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027223+0000", 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032791703999999998 2024-05-08T08:26:38.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027303+0000", 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0068999999999995e-05 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027391+0000", 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7899000000000007e-05 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108445 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.265 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238884256999999, 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033150011, 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108445 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994441+0000", 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994448+0000", 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9929999999999998e-06 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994452+0000", 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8199999999999996e-06 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027420+0000", 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032967989000000003 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027500+0000", 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9332000000000005e-05 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027588+0000", 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8080999999999999e-05 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108446 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238861998000001, 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033416003, 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108446 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994461+0000", 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994463+0000", 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994470+0000", 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4489999999999996e-06 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994475+0000", 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027699+0000", 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033223821000000001 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027779+0000", 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0411e-05 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027876+0000", 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6855999999999996e-05 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.273 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108447 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.238839953999999, 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033598876, 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108447 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.274 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994483+0000", 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:26:38.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994485+0000", 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994492+0000", 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994497+0000", 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3569999999999997e-06 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027912+0000", 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033415331999999999 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027991+0000", 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8825999999999994e-05 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.028081+0000", 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9858000000000006e-05 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108675 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.233844035000001, 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032085805000000002, 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108675 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.278 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999480+0000", 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.793e-06 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999486+0000", 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999490+0000", 2024-05-08T08:26:38.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.464e-06 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031407+0000", 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031916959000000002 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031488+0000", 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0567000000000003e-05 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031564+0000", 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6114999999999999e-05 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108676 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.233826536999999, 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032251646000000002, 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.282 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108676 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999498+0000", 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999503+0000", 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999509+0000", 2024-05-08T08:26:38.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8270000000000003e-06 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031594+0000", 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032084797999999998 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031672+0000", 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8765999999999998e-05 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031747+0000", 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4881000000000004e-05 2024-05-08T08:26:38.285 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108677 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.233807780999999, 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032405850999999999, 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.286 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108677 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999515+0000", 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.79e-07 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999516+0000", 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999522+0000", 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2460000000000003e-06 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999526+0000", 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.673e-06 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031777+0000", 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032250681000000003 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031851+0000", 2024-05-08T08:26:38.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3694000000000003e-05 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031920+0000", 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9449999999999994e-05 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108678 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.233790475999999, 2024-05-08T08:26:38.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032573421999999998, 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108678 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:38.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0800000000000004e-07 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999538+0000", 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-05-08T08:26:38.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999545+0000", 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999549+0000", 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031947+0000", 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032397903999999998 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032029+0000", 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1985999999999997e-05 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032105+0000", 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6032999999999995e-05 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.294 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108703 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6832128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.232463131999999, 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03217747, 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108703 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:38.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000861+0000", 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8199999999999999e-06 2024-05-08T08:26:38.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000866+0000", 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0880000000000002e-06 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000871+0000", 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510000000000004e-06 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032860+0000", 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031989640999999999 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032962+0000", 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010197200000000001 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033036+0000", 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4299000000000001e-05 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108704 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6864896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.298 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.232114028000002, 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032008086999999998, 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108704 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001209+0000", 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5199999999999996e-07 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001213+0000", 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8770000000000003e-06 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.300 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001224+0000", 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1342000000000001e-05 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001231+0000", 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033067+0000", 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031835544 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033149+0000", 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1719999999999997e-05 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033216+0000", 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7645999999999996e-05 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108705 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.232078672, 2024-05-08T08:26:38.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032150629999999999, 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108705 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001244+0000", 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001247+0000", 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001254+0000", 2024-05-08T08:26:38.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7190000000000001e-06 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001260+0000", 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5099999999999998e-06 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033245+0000", 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031984929000000002 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033322+0000", 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7435999999999995e-05 2024-05-08T08:26:38.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033394+0000", 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1740000000000001e-05 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108706 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6930432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.232050979, 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032385262999999997, 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.306 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108706 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:38.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001272+0000", 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001274+0000", 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001281+0000", 2024-05-08T08:26:38.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0840000000000003e-06 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001287+0000", 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033490+0000", 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032203455999999998 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033573+0000", 2024-05-08T08:26:38.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2860000000000003e-05 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033656+0000", 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3380999999999993e-05 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.310 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:38.311 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:38.319+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:38.312 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 102, 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf", 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-05-08T08:24:44.143941+0000", 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-05-08T08:26:38.012635+0000", 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-05-08T08:25:44.118968+0000", 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-05-08T08:24:46.429925+0000", 2024-05-08T08:26:38.363 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5799936, 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-05-08T08:26:38.364 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:55.134472+0000", 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:26:38.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:26:38.366 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "16", 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:26:38.367 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:26:38.368 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:26:38.369 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "error_message": "pool 1 has some PGs where all OSDs are with primary_affinity 0 (at least pg 1.4), read balance score may not be reliable", 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 0, 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 0 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:56.465515+0000", 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:26:38.370 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:26:38.371 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:26:38.372 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:26:38.373 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:26:38.374 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-05-08T08:26:38.375 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.69752693176269531, 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.93000000715255737, 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.69999998807907104 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:25:03.054989+0000", 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:26:38.376 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-05-08T08:26:38.377 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "102", 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:26:38.378 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 39, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 102, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:26:38.379 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:26:38.380 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "error_message": "pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.2), read balance score may not be reliable", 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 0, 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 0 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14afea42-356a-4163-ac3f-2e270e807917", 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 91, 2024-05-08T08:26:38.382 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.383 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.385 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/727789107", 2024-05-08T08:26:38.386 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/727789107", 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/727789107", 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/727789107", 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "db77a923-8f49-4b47-b752-00b475917a0b", 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:38.387 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 13, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 30, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 54, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 91, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 32, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-05-08T08:26:38.388 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.389 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2489759853", 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2489759853", 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2489759853", 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2489759853", 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "3d508691-8e40-4592-82d0-eaf4c4c7d7dd", 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.392 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.582733154296875, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.5802154541015625, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 56, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.393 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.394 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.395 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.396 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2736372115", 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2736372115", 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2736372115", 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2736372115", 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.397 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2545300a-428a-41ed-b19a-3a0cb9e5e755", 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 83, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.398 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.399 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.400 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.401 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/573482428", 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/573482428", 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/573482428", 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/573482428", 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "886aeed5-4495-4064-8579-050d38a4ad1b", 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.403 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 91, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6801", 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6803", 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6805", 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6807", 2024-05-08T08:26:38.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6813", 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6815", 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6809", 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.407 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6811", 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6803/3424463091", 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6807/3424463091", 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6815/3424463091", 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6811/3424463091", 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:26:38.408 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4d15cef0-9ee9-4069-a6c4-a650e873ee03", 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 91, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.409 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6816", 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6818", 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.410 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6820", 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6822", 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.411 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6828", 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6830", 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6824", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6826", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6818/3709461555", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6822/3709461555", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6830/3709461555", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6826/3709461555", 2024-05-08T08:26:38.413 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "910ac664-9573-4851-9e2b-89ac6015fdc1", 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:38.414 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 91, 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6800", 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6802", 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.415 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6804", 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6806", 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6812", 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6814", 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6808", 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6810", 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.418 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6802/405561966", 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6806/405561966", 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6814/405561966", 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6810/405561966", 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14cc2d94-48e4-47c9-896a-9be9281ef0c5", 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:26:38.419 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 91, 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6817", 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6819", 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6821", 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.421 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6823", 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6829", 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6831", 2024-05-08T08:26:38.422 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6825", 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6827", 2024-05-08T08:26:38.423 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6819/570236217", 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6823/570236217", 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6831/570236217", 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6827/570236217", 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-05-08T08:26:38.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.572981+0000", 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.425 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.626543+0000", 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 32 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.638878+0000", 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.599293+0000", 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.427 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.565676+0000", 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.428 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.552124+0000", 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.531428+0000", 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:26:38.429 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.658391+0000", 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.a", 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:26:38.430 INFO:teuthology.orchestra.run.smithi031.stdout: 3, 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: 4 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [], 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [], 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-05-08T08:26:38.431 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 24, 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 11 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 24, 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-05-08T08:26:38.433 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.434 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:38.436 INFO:tasks.thrashosds.thrasher:j is {'epoch': 102, 'fsid': '1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf', 'created': '2024-05-08T08:24:44.143941+0000', 'modified': '2024-05-08T08:26:38.012635+0000', 'last_up_change': '2024-05-08T08:25:44.118968+0000', 'last_in_change': '2024-05-08T08:24:46.429925+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:24:55.134472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 1 has some PGs where all OSDs are with primary_affinity 0 (at least pg 1.4), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:24:56.465515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.6975269317626953, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.699999988079071}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:25:03.054989+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 102, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.2), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}], 'osds': [{'osd': 0, 'uuid': '14afea42-356a-4163-ac3f-2e270e807917', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 727789107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 727789107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 727789107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 727789107}]}, 'public_addr': '172.21.15.31:6801/727789107', 'cluster_addr': '172.21.15.31:6803/727789107', 'heartbeat_back_addr': '172.21.15.31:6807/727789107', 'heartbeat_front_addr': '172.21.15.31:6805/727789107', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'db77a923-8f49-4b47-b752-00b475917a0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 30, 'up_from': 54, 'up_thru': 91, 'down_at': 32, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2489759853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2489759853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2489759853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2489759853}]}, 'public_addr': '172.21.15.31:6817/2489759853', 'cluster_addr': '172.21.15.31:6819/2489759853', 'heartbeat_back_addr': '172.21.15.31:6823/2489759853', 'heartbeat_front_addr': '172.21.15.31:6821/2489759853', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '3d508691-8e40-4592-82d0-eaf4c4c7d7dd', 'up': 1, 'in': 1, 'weight': 0.582733154296875, 'primary_affinity': 0.5802154541015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 56, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2736372115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2736372115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2736372115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2736372115}]}, 'public_addr': '172.21.15.31:6809/2736372115', 'cluster_addr': '172.21.15.31:6811/2736372115', 'heartbeat_back_addr': '172.21.15.31:6815/2736372115', 'heartbeat_front_addr': '172.21.15.31:6813/2736372115', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2545300a-428a-41ed-b19a-3a0cb9e5e755', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 83, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 573482428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 573482428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 573482428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 573482428}]}, 'public_addr': '172.21.15.31:6825/573482428', 'cluster_addr': '172.21.15.31:6827/573482428', 'heartbeat_back_addr': '172.21.15.31:6831/573482428', 'heartbeat_front_addr': '172.21.15.31:6829/573482428', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '886aeed5-4495-4064-8579-050d38a4ad1b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6801', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6803', 'nonce': 3424463091}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6805', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6807', 'nonce': 3424463091}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6813', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6815', 'nonce': 3424463091}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6809', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6811', 'nonce': 3424463091}]}, 'public_addr': '172.21.15.149:6803/3424463091', 'cluster_addr': '172.21.15.149:6807/3424463091', 'heartbeat_back_addr': '172.21.15.149:6815/3424463091', 'heartbeat_front_addr': '172.21.15.149:6811/3424463091', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4d15cef0-9ee9-4069-a6c4-a650e873ee03', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6816', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6818', 'nonce': 3709461555}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6820', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6822', 'nonce': 3709461555}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6828', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6830', 'nonce': 3709461555}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6824', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6826', 'nonce': 3709461555}]}, 'public_addr': '172.21.15.149:6818/3709461555', 'cluster_addr': '172.21.15.149:6822/3709461555', 'heartbeat_back_addr': '172.21.15.149:6830/3709461555', 'heartbeat_front_addr': '172.21.15.149:6826/3709461555', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '910ac664-9573-4851-9e2b-89ac6015fdc1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6800', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6802', 'nonce': 405561966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6804', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6806', 'nonce': 405561966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6812', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6814', 'nonce': 405561966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6808', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6810', 'nonce': 405561966}]}, 'public_addr': '172.21.15.149:6802/405561966', 'cluster_addr': '172.21.15.149:6806/405561966', 'heartbeat_back_addr': '172.21.15.149:6814/405561966', 'heartbeat_front_addr': '172.21.15.149:6810/405561966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '14cc2d94-48e4-47c9-896a-9be9281ef0c5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6817', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6819', 'nonce': 570236217}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6821', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6823', 'nonce': 570236217}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6829', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6831', 'nonce': 570236217}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6825', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6827', 'nonce': 570236217}]}, 'public_addr': '172.21.15.149:6819/570236217', 'cluster_addr': '172.21.15.149:6823/570236217', 'heartbeat_back_addr': '172.21.15.149:6831/570236217', 'heartbeat_front_addr': '172.21.15.149:6827/570236217', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.572981+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.626543+0000', 'dead_epoch': 32}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.638878+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.599293+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.565676+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.552124+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.531428+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.658391+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.a', 'osds': [3, 4]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 11}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 24, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:26:38.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:26:38.436 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:38.443+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:38.463 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:38.463 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:38.464 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:38.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:38.536 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:38.544+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:38.624 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:38.624 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:38.624 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:38.624 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:38.624 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:38.633 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:38.637 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:38.644+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:38.737 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:38.745+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:38.744 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:38.744 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:38.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:38.744 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69714 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930259233000001, 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046246284999999998, 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69714 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.745 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820801+0000", 2024-05-08T08:26:38.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820803+0000", 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820809+0000", 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7410000000000003e-06 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820815+0000", 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6200000000000004e-06 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866947+0000", 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046131923999999998 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866993+0000", 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867046+0000", 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3052e-05 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69715 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3162112~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930237077999998, 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046369965999999999, 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.749 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69715 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820823+0000", 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820825+0000", 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.655e-06 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820832+0000", 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0849999999999999e-06 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820838+0000", 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867075+0000", 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046236823000000003 2024-05-08T08:26:38.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867133+0000", 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867192+0000", 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9475999999999998e-05 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69716 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3391488~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930213578, 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046544169000000003, 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69716 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820848+0000", 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820855+0000", 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140000000000002e-06 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820861+0000", 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:26:38.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867234+0000", 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046372732 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867299+0000", 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5485000000000001e-05 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867390+0000", 2024-05-08T08:26:38.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969000000000003e-05 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69717 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3620864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930191583000003, 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046712535999999999, 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.758 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69717 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:38.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820870+0000", 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1320000000000001e-06 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820879+0000", 2024-05-08T08:26:38.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820892+0000", 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3356000000000001e-05 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867422+0000", 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046529981999999998 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867485+0000", 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2614999999999996e-05 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867580+0000", 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5359000000000006e-05 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69718 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:38.762 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930158722000002, 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046863712000000002, 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69718 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:38.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820904+0000", 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820911+0000", 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820917+0000", 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867609+0000", 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046691734999999998 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867681+0000", 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1931e-05 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867764+0000", 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3356000000000006e-05 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69719 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4079616~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930135010999997, 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047037906999999997, 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69719 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.767 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820925+0000", 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820927+0000", 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820934+0000", 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.764e-06 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820939+0000", 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.553e-06 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867796+0000", 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046856496999999997 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867868+0000", 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2163999999999996e-05 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867962+0000", 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4337000000000002e-05 2024-05-08T08:26:38.770 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69720 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4308992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930112864999998, 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047192222999999998, 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.771 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69720 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820949+0000", 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820956+0000", 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820963+0000", 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867988+0000", 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047025083000000002 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868053+0000", 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4871999999999998e-05 2024-05-08T08:26:38.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868139+0000", 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5626999999999999e-05 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69721 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4538368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930088464000001, 2024-05-08T08:26:38.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047345685999999998, 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69721 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820974+0000", 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820982+0000", 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8370000000000003e-06 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820988+0000", 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868167+0000", 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047179062000000001 2024-05-08T08:26:38.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868232+0000", 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4889999999999997e-05 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868317+0000", 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4244000000000002e-05 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69722 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4767744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.779 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930063783000001, 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047510657999999997, 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69722 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:38.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821004+0000", 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821011+0000", 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6510000000000001e-06 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821019+0000", 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868347+0000", 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047327859 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868419+0000", 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2725999999999995e-05 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868506+0000", 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6793999999999996e-05 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69723 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4997120~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930032265999998, 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047669162000000001, 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69723 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.784 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821028+0000", 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:26:38.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821030+0000", 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821037+0000", 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821043+0000", 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:26:38.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868536+0000", 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047493103000000002 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868611+0000", 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4647999999999994e-05 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868696+0000", 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5476999999999995e-05 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.787 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69724 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5226496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.930008987000001, 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819591000000002, 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69724 2024-05-08T08:26:38.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821051+0000", 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-05-08T08:26:38.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821053+0000", 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821060+0000", 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.097e-06 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821066+0000", 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:38.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868722+0000", 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047656233999999999 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868795+0000", 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2621999999999997e-05 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868870+0000", 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5352999999999995e-05 2024-05-08T08:26:38.791 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69725 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5455872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929986694999997, 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047973017999999999, 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.792 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69725 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:38.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821075+0000", 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.356e-06 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821082+0000", 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821088+0000", 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868907+0000", 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819382000000001 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868979+0000", 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1922999999999996e-05 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869046+0000", 2024-05-08T08:26:38.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6724999999999996e-05 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69726 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5685248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929947272, 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048091651999999999, 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69726 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821113+0000", 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821115+0000", 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7889999999999999e-06 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821124+0000", 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.191e-06 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821130+0000", 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869065+0000", 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047935187999999997 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869129+0000", 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4356999999999994e-05 2024-05-08T08:26:38.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869204+0000", 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399000000000004e-05 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69727 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5914624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929921165000003, 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048288931, 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.800 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69727 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821139+0000", 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821141+0000", 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821148+0000", 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821153+0000", 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869256+0000", 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048102180000000001 2024-05-08T08:26:38.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869343+0000", 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7828e-05 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869427+0000", 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3666999999999997e-05 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.804 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69728 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6144000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929898717, 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048487729, 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69728 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:38.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821164+0000", 2024-05-08T08:26:38.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821171+0000", 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821176+0000", 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869489+0000", 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048312213999999999 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869566+0000", 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7657999999999994e-05 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869648+0000", 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2194000000000006e-05 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69729 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6373376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929875795000001, 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048680186, 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69729 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.809 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821186+0000", 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821192+0000", 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821197+0000", 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869699+0000", 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048501191999999999 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869782+0000", 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2973000000000007e-05 2024-05-08T08:26:38.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869864+0000", 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2241000000000001e-05 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69730 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6602752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929855597, 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048837093999999998, 2024-05-08T08:26:38.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69730 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821206+0000", 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821212+0000", 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821218+0000", 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869928+0000", 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048710003000000002 2024-05-08T08:26:38.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869995+0000", 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7125000000000005e-05 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870041+0000", 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6134999999999998e-05 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.817 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69731 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6832128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929834524999997, 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048956353000000001, 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69731 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821228+0000", 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821234+0000", 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821239+0000", 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.485e-06 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870066+0000", 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048826621000000001 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870133+0000", 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870181+0000", 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8553000000000001e-05 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69732 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7061504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929795636000001, 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049133148000000001, 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69732 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821267+0000", 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821286+0000", 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9122000000000001e-05 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821292+0000", 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870231+0000", 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048938161000000001 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870316+0000", 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5143999999999997e-05 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870397+0000", 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1118000000000005e-05 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69733 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7290880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 36.929758311000001, 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049266982000000001, 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69733 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.826 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821302+0000", 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:26:38.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821304+0000", 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821311+0000", 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729999999999998e-06 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821317+0000", 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2660000000000001e-06 2024-05-08T08:26:38.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870426+0000", 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049109685 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870497+0000", 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0860999999999998e-05 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870568+0000", 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0984999999999999e-05 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:38.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:38.830 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:38.831 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:38.832 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:26:38.832 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:26:38.834 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-05-08T08:26:38.437136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":888293626,"num_objects":1138,"num_object_clones":261,"num_object_copies":2276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1138,"num_whiteouts":492,"num_read":151837,"num_read_kb":856097,"num_write":20801,"num_write_kb":474217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":27477790,"num_keys_recovered":57,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":16640,"ondisk_log_size":16640,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":392768,"kb_used_data":56760,"kb_used_omap":24,"kb_used_meta":335655,"kb_avail":754581952,"statfs":{"total":773094113280,"available":772691918848,"internally_reserved":0,"allocated":58122240,"data_stored":538351925,"data_compressed":17025836,"data_compressed_allocated":41066496,"data_compressed_original":529364121,"omap_allocated":24704,"internal_metadata":343711616},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-102886668,"num_objects":-284,"num_object_clones":-37,"num_object_copies":-568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-284,"num_whiteouts":-35,"num_read":1116,"num_read_kb":960,"num_write":371,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000575"},"pg_stats":[{"pgid":"3.1f","version":"99'659","reported_seq":4569,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.162303+0000","last_change":"2024-05-08T08:26:35.014361+0000","last_active":"2024-05-08T08:26:35.162303+0000","last_peered":"2024-05-08T08:26:35.162303+0000","last_clean":"2024-05-08T08:26:35.162303+0000","last_became_active":"2024-05-08T08:25:04.610836+0000","last_became_peered":"2024-05-08T08:25:04.610836+0000","last_unstale":"2024-05-08T08:26:35.162303+0000","last_undegraded":"2024-05-08T08:26:35.162303+0000","last_fullsized":"2024-05-08T08:26:35.162303+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:59:53.814146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000864229,"stat_sum":{"num_bytes":20581408,"num_objects":31,"num_object_clones":6,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":15,"num_read":7979,"num_read_kb":28266,"num_write":675,"num_write_kb":7014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1e","version":"99'748","reported_seq":3492,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.130670+0000","last_change":"2024-05-08T08:26:35.011581+0000","last_active":"2024-05-08T08:26:35.130670+0000","last_peered":"2024-05-08T08:26:35.130670+0000","last_clean":"2024-05-08T08:26:35.130670+0000","last_became_active":"2024-05-08T08:26:28.402388+0000","last_became_peered":"2024-05-08T08:26:28.402388+0000","last_unstale":"2024-05-08T08:26:35.130670+0000","last_undegraded":"2024-05-08T08:26:35.130670+0000","last_fullsized":"2024-05-08T08:26:35.130670+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:15.741443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097886700000000002,"stat_sum":{"num_bytes":19774911,"num_objects":48,"num_object_clones":13,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":24,"num_read":5243,"num_read_kb":27094,"num_write":1173,"num_write_kb":17969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1d","version":"99'514","reported_seq":1954,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.097031+0000","last_change":"2024-05-08T08:26:35.000145+0000","last_active":"2024-05-08T08:26:35.097031+0000","last_peered":"2024-05-08T08:26:35.097031+0000","last_clean":"2024-05-08T08:26:35.097031+0000","last_became_active":"2024-05-08T08:26:04.254033+0000","last_became_peered":"2024-05-08T08:26:04.254033+0000","last_unstale":"2024-05-08T08:26:35.097031+0000","last_undegraded":"2024-05-08T08:26:35.097031+0000","last_fullsized":"2024-05-08T08:26:35.097031+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":514,"log_dups_size":0,"ondisk_log_size":514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:26.861671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067060700000000004,"stat_sum":{"num_bytes":14449780,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":2704,"num_read_kb":21948,"num_write":681,"num_write_kb":14048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1c","version":"99'724","reported_seq":7459,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.069010+0000","last_change":"2024-05-08T08:26:35.000349+0000","last_active":"2024-05-08T08:26:35.069010+0000","last_peered":"2024-05-08T08:26:35.069010+0000","last_clean":"2024-05-08T08:26:35.069010+0000","last_became_active":"2024-05-08T08:26:04.253888+0000","last_became_peered":"2024-05-08T08:26:04.253888+0000","last_unstale":"2024-05-08T08:26:35.069010+0000","last_undegraded":"2024-05-08T08:26:35.069010+0000","last_fullsized":"2024-05-08T08:26:35.069010+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:53:21.021371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087898399999999997,"stat_sum":{"num_bytes":94835931,"num_objects":32,"num_object_clones":7,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":13,"num_read":9443,"num_read_kb":74407,"num_write":924,"num_write_kb":23727,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1b","version":"99'1038","reported_seq":2376,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.099223+0000","last_change":"2024-05-08T08:26:35.014683+0000","last_active":"2024-05-08T08:26:35.099223+0000","last_peered":"2024-05-08T08:26:35.099223+0000","last_clean":"2024-05-08T08:26:35.099223+0000","last_became_active":"2024-05-08T08:25:04.612215+0000","last_became_peered":"2024-05-08T08:25:04.612215+0000","last_unstale":"2024-05-08T08:26:35.099223+0000","last_undegraded":"2024-05-08T08:26:35.099223+0000","last_fullsized":"2024-05-08T08:26:35.099223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1038,"log_dups_size":0,"ondisk_log_size":1038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:02.664831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010247940000000001,"stat_sum":{"num_bytes":18110614,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":21,"num_read":5902,"num_read_kb":12377,"num_write":1664,"num_write_kb":13429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1a","version":"99'415","reported_seq":12278,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.147448+0000","last_change":"2024-05-08T08:26:34.999816+0000","last_active":"2024-05-08T08:26:35.147448+0000","last_peered":"2024-05-08T08:26:35.147448+0000","last_clean":"2024-05-08T08:26:35.147448+0000","last_became_active":"2024-05-08T08:25:48.157597+0000","last_became_peered":"2024-05-08T08:25:48.157597+0000","last_unstale":"2024-05-08T08:26:35.147448+0000","last_undegraded":"2024-05-08T08:26:35.147448+0000","last_fullsized":"2024-05-08T08:26:35.147448+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:37:19.037381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045705399999999998,"stat_sum":{"num_bytes":45666384,"num_objects":39,"num_object_clones":7,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":17,"num_read":11898,"num_read_kb":36955,"num_write":461,"num_write_kb":15894,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.19","version":"102'391","reported_seq":2009,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T08:26:38.116113+0000","last_change":"2024-05-08T08:26:38.020082+0000","last_active":"2024-05-08T08:26:38.116113+0000","last_peered":"2024-05-08T08:26:38.116113+0000","last_clean":"2024-05-08T08:26:38.116113+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:26:38.116113+0000","last_undegraded":"2024-05-08T08:26:38.116113+0000","last_fullsized":"2024-05-08T08:26:38.116113+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":391,"log_dups_size":0,"ondisk_log_size":391,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:38:53.271965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070087100000000005,"stat_sum":{"num_bytes":14811282,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":12,"num_read":2333,"num_read_kb":23330,"num_write":443,"num_write_kb":21316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"b"}]},{"pgid":"3.18","version":"99'352","reported_seq":12431,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.095558+0000","last_change":"2024-05-08T08:26:35.013924+0000","last_active":"2024-05-08T08:26:35.095558+0000","last_peered":"2024-05-08T08:26:35.095558+0000","last_clean":"2024-05-08T08:26:35.095558+0000","last_became_active":"2024-05-08T08:26:28.398428+0000","last_became_peered":"2024-05-08T08:26:28.398428+0000","last_unstale":"2024-05-08T08:26:35.095558+0000","last_undegraded":"2024-05-08T08:26:35.095558+0000","last_fullsized":"2024-05-08T08:26:35.095558+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":352,"log_dups_size":0,"ondisk_log_size":352,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:16:31.500629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059946100000000003,"stat_sum":{"num_bytes":11945945,"num_objects":29,"num_object_clones":7,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":11,"num_read":12441,"num_read_kb":31299,"num_write":353,"num_write_kb":16473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.17","version":"99'707","reported_seq":2023,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.160511+0000","last_change":"2024-05-08T08:26:35.014482+0000","last_active":"2024-05-08T08:26:35.160511+0000","last_peered":"2024-05-08T08:26:35.160511+0000","last_clean":"2024-05-08T08:26:35.160511+0000","last_became_active":"2024-05-08T08:25:04.610818+0000","last_became_peered":"2024-05-08T08:25:04.610818+0000","last_unstale":"2024-05-08T08:26:35.160511+0000","last_undegraded":"2024-05-08T08:26:35.160511+0000","last_fullsized":"2024-05-08T08:26:35.160511+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":707,"log_dups_size":0,"ondisk_log_size":707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:58:39.446509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096086800000000003,"stat_sum":{"num_bytes":30029466,"num_objects":35,"num_object_clones":4,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":13,"num_read":3970,"num_read_kb":7667,"num_write":1010,"num_write_kb":8238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.16","version":"99'321","reported_seq":1682,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.097297+0000","last_change":"2024-05-08T08:26:35.091382+0000","last_active":"2024-05-08T08:26:35.097297+0000","last_peered":"2024-05-08T08:26:35.097297+0000","last_clean":"2024-05-08T08:26:35.097297+0000","last_became_active":"2024-05-08T08:26:04.253934+0000","last_became_peered":"2024-05-08T08:26:04.253934+0000","last_unstale":"2024-05-08T08:26:35.097297+0000","last_undegraded":"2024-05-08T08:26:35.097297+0000","last_fullsized":"2024-05-08T08:26:35.097297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":321,"log_dups_size":0,"ondisk_log_size":321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:52:13.206257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002155536,"stat_sum":{"num_bytes":10913207,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":16,"num_read":1967,"num_read_kb":14258,"num_write":317,"num_write_kb":11766,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.15","version":"99'354","reported_seq":3254,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.105373+0000","last_change":"2024-05-08T08:26:35.090368+0000","last_active":"2024-05-08T08:26:35.105373+0000","last_peered":"2024-05-08T08:26:35.105373+0000","last_clean":"2024-05-08T08:26:35.105373+0000","last_became_active":"2024-05-08T08:25:04.606180+0000","last_became_peered":"2024-05-08T08:25:04.606180+0000","last_unstale":"2024-05-08T08:26:35.105373+0000","last_undegraded":"2024-05-08T08:26:35.105373+0000","last_fullsized":"2024-05-08T08:26:35.105373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":354,"log_dups_size":0,"ondisk_log_size":354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:57.392657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012031399999999999,"stat_sum":{"num_bytes":30398012,"num_objects":34,"num_object_clones":8,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":3791,"num_read_kb":18054,"num_write":382,"num_write_kb":5770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.14","version":"99'466","reported_seq":1299,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.120847+0000","last_change":"2024-05-08T08:26:35.000379+0000","last_active":"2024-05-08T08:26:35.120847+0000","last_peered":"2024-05-08T08:26:35.120847+0000","last_clean":"2024-05-08T08:26:35.120847+0000","last_became_active":"2024-05-08T08:25:04.614760+0000","last_became_peered":"2024-05-08T08:25:04.614760+0000","last_unstale":"2024-05-08T08:26:35.120847+0000","last_undegraded":"2024-05-08T08:26:35.120847+0000","last_fullsized":"2024-05-08T08:26:35.120847+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":466,"log_dups_size":0,"ondisk_log_size":466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:45:14.556987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070229999999999999,"stat_sum":{"num_bytes":15590732,"num_objects":36,"num_object_clones":11,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":1554,"num_read_kb":8783,"num_write":629,"num_write_kb":10686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"1.7","version":"0'0","reported_seq":178,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.010521+0000","last_change":"2024-05-08T08:24:56.564123+0000","last_active":"2024-05-08T08:26:35.010521+0000","last_peered":"2024-05-08T08:26:35.010521+0000","last_clean":"2024-05-08T08:26:35.010521+0000","last_became_active":"2024-05-08T08:24:56.563865+0000","last_became_peered":"2024-05-08T08:24:56.563865+0000","last_unstale":"2024-05-08T08:26:35.010521+0000","last_undegraded":"2024-05-08T08:26:35.010521+0000","last_fullsized":"2024-05-08T08:26:35.010521+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:32.338505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"99'383","reported_seq":8915,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.134649+0000","last_change":"2024-05-08T08:26:35.005493+0000","last_active":"2024-05-08T08:26:35.134649+0000","last_peered":"2024-05-08T08:26:35.134649+0000","last_clean":"2024-05-08T08:26:35.134649+0000","last_became_active":"2024-05-08T08:26:28.402256+0000","last_became_peered":"2024-05-08T08:26:28.402256+0000","last_unstale":"2024-05-08T08:26:35.134649+0000","last_undegraded":"2024-05-08T08:26:35.134649+0000","last_fullsized":"2024-05-08T08:26:35.134649+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":383,"log_dups_size":0,"ondisk_log_size":383,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:10.280895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001140662,"stat_sum":{"num_bytes":16728566,"num_objects":47,"num_object_clones":14,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":20,"num_read":8936,"num_read_kb":19925,"num_write":373,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"1.6","version":"0'0","reported_seq":178,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.010573+0000","last_change":"2024-05-08T08:24:56.680294+0000","last_active":"2024-05-08T08:26:35.010573+0000","last_peered":"2024-05-08T08:26:35.010573+0000","last_clean":"2024-05-08T08:26:35.010573+0000","last_became_active":"2024-05-08T08:24:56.679982+0000","last_became_peered":"2024-05-08T08:24:56.679982+0000","last_unstale":"2024-05-08T08:26:35.010573+0000","last_undegraded":"2024-05-08T08:26:35.010573+0000","last_fullsized":"2024-05-08T08:26:35.010573+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:18:54.867771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"102'403","reported_seq":5139,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T08:26:38.121734+0000","last_change":"2024-05-08T08:26:38.019703+0000","last_active":"2024-05-08T08:26:38.121734+0000","last_peered":"2024-05-08T08:26:38.121734+0000","last_clean":"2024-05-08T08:26:38.121734+0000","last_became_active":"2024-05-08T08:26:01.241372+0000","last_became_peered":"2024-05-08T08:26:01.241372+0000","last_unstale":"2024-05-08T08:26:38.121734+0000","last_undegraded":"2024-05-08T08:26:38.121734+0000","last_fullsized":"2024-05-08T08:26:38.121734+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:19:32.523625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078091399999999996,"stat_sum":{"num_bytes":35413881,"num_objects":33,"num_object_clones":10,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5580,"num_read_kb":36769,"num_write":463,"num_write_kb":6639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"b"}]},{"pgid":"1.5","version":"0'0","reported_seq":181,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:34.999285+0000","last_change":"2024-05-08T08:25:58.223928+0000","last_active":"2024-05-08T08:26:34.999285+0000","last_peered":"2024-05-08T08:26:34.999285+0000","last_clean":"2024-05-08T08:26:34.999285+0000","last_became_active":"2024-05-08T08:25:58.223356+0000","last_became_peered":"2024-05-08T08:25:58.223356+0000","last_unstale":"2024-05-08T08:26:34.999285+0000","last_undegraded":"2024-05-08T08:26:34.999285+0000","last_fullsized":"2024-05-08T08:26:34.999285+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:12:12.674757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"99'742","reported_seq":7450,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.115311+0000","last_change":"2024-05-08T08:26:35.092125+0000","last_active":"2024-05-08T08:26:35.115311+0000","last_peered":"2024-05-08T08:26:35.115311+0000","last_clean":"2024-05-08T08:26:35.115311+0000","last_became_active":"2024-05-08T08:26:28.403209+0000","last_became_peered":"2024-05-08T08:26:28.403209+0000","last_unstale":"2024-05-08T08:26:35.115311+0000","last_undegraded":"2024-05-08T08:26:35.115311+0000","last_fullsized":"2024-05-08T08:26:35.115311+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":742,"log_dups_size":0,"ondisk_log_size":742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:09:17.586609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024226899999999999,"stat_sum":{"num_bytes":44612987,"num_objects":49,"num_object_clones":11,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":17,"num_read":10126,"num_read_kb":75093,"num_write":995,"num_write_kb":39588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"1.4","version":"0'0","reported_seq":177,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:34.998904+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:26:34.998904+0000","last_peered":"2024-05-08T08:26:34.998904+0000","last_clean":"2024-05-08T08:26:34.998904+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:26:34.998904+0000","last_undegraded":"2024-05-08T08:26:34.998904+0000","last_fullsized":"2024-05-08T08:26:34.998904+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"99'430","reported_seq":2880,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.119959+0000","last_change":"2024-05-08T08:26:35.014644+0000","last_active":"2024-05-08T08:26:35.119959+0000","last_peered":"2024-05-08T08:26:35.119959+0000","last_clean":"2024-05-08T08:26:35.119959+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:26:35.119959+0000","last_undegraded":"2024-05-08T08:26:35.119959+0000","last_fullsized":"2024-05-08T08:26:35.119959+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":430,"log_dups_size":0,"ondisk_log_size":430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:44:20.532587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011396659999999999,"stat_sum":{"num_bytes":35575901,"num_objects":36,"num_object_clones":8,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":17,"num_read":3451,"num_read_kb":39796,"num_write":425,"num_write_kb":20003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.1","version":"98'1175","reported_seq":2671,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.013584+0000","last_change":"2024-05-08T08:26:35.013834+0000","last_active":"2024-05-08T08:26:35.013584+0000","last_peered":"2024-05-08T08:26:35.013584+0000","last_clean":"2024-05-08T08:26:35.013584+0000","last_became_active":"2024-05-08T08:25:04.615077+0000","last_became_peered":"2024-05-08T08:25:04.615077+0000","last_unstale":"2024-05-08T08:26:35.013584+0000","last_undegraded":"2024-05-08T08:26:35.013584+0000","last_fullsized":"2024-05-08T08:26:35.013584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1175,"log_dups_size":0,"ondisk_log_size":1175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:23:08.874123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053456900000000004,"stat_sum":{"num_bytes":11737820,"num_objects":48,"num_object_clones":12,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":25,"num_read":6669,"num_read_kb":16069,"num_write":1838,"num_write_kb":9312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"2.0","version":"17'117","reported_seq":333,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.088966+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:26:35.088966+0000","last_peered":"2024-05-08T08:26:35.088966+0000","last_clean":"2024-05-08T08:26:35.088966+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:26:35.088966+0000","last_undegraded":"2024-05-08T08:26:35.088966+0000","last_fullsized":"2024-05-08T08:26:35.088966+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":114,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T08:26:38.020585+0000","last_change":"2024-05-08T08:25:45.225724+0000","last_active":"2024-05-08T08:26:38.020585+0000","last_peered":"2024-05-08T08:26:38.020585+0000","last_clean":"2024-05-08T08:26:38.020585+0000","last_became_active":"2024-05-08T08:25:45.225388+0000","last_became_peered":"2024-05-08T08:25:45.225388+0000","last_unstale":"2024-05-08T08:26:38.020585+0000","last_undegraded":"2024-05-08T08:26:38.020585+0000","last_fullsized":"2024-05-08T08:26:38.020585+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:08:54.551477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"102'336","reported_seq":1652,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T08:26:38.127351+0000","last_change":"2024-05-08T08:26:38.020334+0000","last_active":"2024-05-08T08:26:38.127351+0000","last_peered":"2024-05-08T08:26:38.127351+0000","last_clean":"2024-05-08T08:26:38.127351+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:26:38.127351+0000","last_undegraded":"2024-05-08T08:26:38.127351+0000","last_fullsized":"2024-05-08T08:26:38.127351+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":336,"log_dups_size":0,"ondisk_log_size":336,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:12:58.011287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011250920000000001,"stat_sum":{"num_bytes":22075631,"num_objects":37,"num_object_clones":8,"num_object_copies":74,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37,"num_whiteouts":18,"num_read":1041,"num_read_kb":15366,"num_write":365,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"b"}]},{"pgid":"1.2","version":"0'0","reported_seq":178,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.088982+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:26:35.088982+0000","last_peered":"2024-05-08T08:26:35.088982+0000","last_clean":"2024-05-08T08:26:35.088982+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:26:35.088982+0000","last_undegraded":"2024-05-08T08:26:35.088982+0000","last_fullsized":"2024-05-08T08:26:35.088982+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"99'677","reported_seq":5651,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.140751+0000","last_change":"2024-05-08T08:26:35.000614+0000","last_active":"2024-05-08T08:26:35.140751+0000","last_peered":"2024-05-08T08:26:35.140751+0000","last_clean":"2024-05-08T08:26:35.140751+0000","last_became_active":"2024-05-08T08:25:04.611900+0000","last_became_peered":"2024-05-08T08:25:04.611900+0000","last_unstale":"2024-05-08T08:26:35.140751+0000","last_undegraded":"2024-05-08T08:26:35.140751+0000","last_fullsized":"2024-05-08T08:26:35.140751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:44:50.530476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072625599999999999,"stat_sum":{"num_bytes":19631352,"num_objects":30,"num_object_clones":7,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":12,"num_read":6531,"num_read_kb":37028,"num_write":763,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.013238+0000","last_change":"2024-05-08T08:26:04.255319+0000","last_active":"2024-05-08T08:26:35.013238+0000","last_peered":"2024-05-08T08:26:35.013238+0000","last_clean":"2024-05-08T08:26:35.013238+0000","last_became_active":"2024-05-08T08:26:04.255074+0000","last_became_peered":"2024-05-08T08:26:04.255074+0000","last_unstale":"2024-05-08T08:26:35.013238+0000","last_undegraded":"2024-05-08T08:26:35.013238+0000","last_fullsized":"2024-05-08T08:26:35.013238+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:52:16.514474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"99'295","reported_seq":1703,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:34.999393+0000","last_change":"2024-05-08T08:26:34.999393+0000","last_active":"2024-05-08T08:26:34.999393+0000","last_peered":"2024-05-08T08:26:34.999393+0000","last_clean":"2024-05-08T08:26:34.999393+0000","last_became_active":"2024-05-08T08:25:04.606801+0000","last_became_peered":"2024-05-08T08:25:04.606801+0000","last_unstale":"2024-05-08T08:26:34.999393+0000","last_undegraded":"2024-05-08T08:26:34.999393+0000","last_fullsized":"2024-05-08T08:26:34.999393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":295,"log_dups_size":0,"ondisk_log_size":295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:12:36.488236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058465199999999996,"stat_sum":{"num_bytes":61920517,"num_objects":40,"num_object_clones":10,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":21,"num_read":1233,"num_read_kb":48447,"num_write":284,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"1.0","version":"0'0","reported_seq":178,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.088874+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:26:35.088874+0000","last_peered":"2024-05-08T08:26:35.088874+0000","last_clean":"2024-05-08T08:26:35.088874+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:26:35.088874+0000","last_undegraded":"2024-05-08T08:26:35.088874+0000","last_fullsized":"2024-05-08T08:26:35.088874+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"102'386","reported_seq":4465,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T08:26:38.124926+0000","last_change":"2024-05-08T08:26:38.019588+0000","last_active":"2024-05-08T08:26:38.124926+0000","last_peered":"2024-05-08T08:26:38.124926+0000","last_clean":"2024-05-08T08:26:38.124926+0000","last_became_active":"2024-05-08T08:26:28.410649+0000","last_became_peered":"2024-05-08T08:26:28.410649+0000","last_unstale":"2024-05-08T08:26:38.124926+0000","last_undegraded":"2024-05-08T08:26:38.124926+0000","last_fullsized":"2024-05-08T08:26:38.124926+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:44:16.580632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053978800000000003,"stat_sum":{"num_bytes":21621744,"num_objects":36,"num_object_clones":7,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":15,"num_read":5464,"num_read_kb":26131,"num_write":402,"num_write_kb":20632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"18","length":"b"}]},{"pgid":"3.9","version":"99'310","reported_seq":1406,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.114895+0000","last_change":"2024-05-08T08:26:35.000110+0000","last_active":"2024-05-08T08:26:35.114895+0000","last_peered":"2024-05-08T08:26:35.114895+0000","last_clean":"2024-05-08T08:26:35.114895+0000","last_became_active":"2024-05-08T08:25:04.901435+0000","last_became_peered":"2024-05-08T08:25:04.901435+0000","last_unstale":"2024-05-08T08:26:35.114895+0000","last_undegraded":"2024-05-08T08:26:35.114895+0000","last_fullsized":"2024-05-08T08:26:35.114895+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":310,"log_dups_size":0,"ondisk_log_size":310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:07:17.085040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074946300000000002,"stat_sum":{"num_bytes":11085840,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":16,"num_read":2199,"num_read_kb":7290,"num_write":303,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.a","version":"99'597","reported_seq":1784,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.107897+0000","last_change":"2024-05-08T08:26:34.999798+0000","last_active":"2024-05-08T08:26:35.107897+0000","last_peered":"2024-05-08T08:26:35.107897+0000","last_clean":"2024-05-08T08:26:35.107897+0000","last_became_active":"2024-05-08T08:26:28.398936+0000","last_became_peered":"2024-05-08T08:26:28.398936+0000","last_unstale":"2024-05-08T08:26:35.107897+0000","last_undegraded":"2024-05-08T08:26:35.107897+0000","last_fullsized":"2024-05-08T08:26:35.107897+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":597,"log_dups_size":0,"ondisk_log_size":597,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:56:32.768005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050674599999999998,"stat_sum":{"num_bytes":28282536,"num_objects":39,"num_object_clones":8,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":39,"num_whiteouts":13,"num_read":2160,"num_read_kb":18892,"num_write":770,"num_write_kb":10100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.b","version":"99'334","reported_seq":2323,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.114252+0000","last_change":"2024-05-08T08:26:35.014202+0000","last_active":"2024-05-08T08:26:35.114252+0000","last_peered":"2024-05-08T08:26:35.114252+0000","last_clean":"2024-05-08T08:26:35.114252+0000","last_became_active":"2024-05-08T08:26:28.399037+0000","last_became_peered":"2024-05-08T08:26:28.399037+0000","last_unstale":"2024-05-08T08:26:35.114252+0000","last_undegraded":"2024-05-08T08:26:35.114252+0000","last_fullsized":"2024-05-08T08:26:35.114252+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":334,"log_dups_size":0,"ondisk_log_size":334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:08:26.460247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00070454399999999998,"stat_sum":{"num_bytes":34120901,"num_objects":44,"num_object_clones":8,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":18,"num_read":1809,"num_read_kb":17243,"num_write":314,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.c","version":"99'362","reported_seq":4316,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.107950+0000","last_change":"2024-05-08T08:26:35.004606+0000","last_active":"2024-05-08T08:26:35.107950+0000","last_peered":"2024-05-08T08:26:35.107950+0000","last_clean":"2024-05-08T08:26:35.107950+0000","last_became_active":"2024-05-08T08:26:28.402309+0000","last_became_peered":"2024-05-08T08:26:28.402309+0000","last_unstale":"2024-05-08T08:26:35.107950+0000","last_undegraded":"2024-05-08T08:26:35.107950+0000","last_fullsized":"2024-05-08T08:26:35.107950+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:11:59.547627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035186700000000001,"stat_sum":{"num_bytes":58995571,"num_objects":18,"num_object_clones":2,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":9,"num_read":4485,"num_read_kb":41622,"num_write":406,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.d","version":"99'346","reported_seq":1620,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.110557+0000","last_change":"2024-05-08T08:26:35.091380+0000","last_active":"2024-05-08T08:26:35.110557+0000","last_peered":"2024-05-08T08:26:35.110557+0000","last_clean":"2024-05-08T08:26:35.110557+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:26:35.110557+0000","last_undegraded":"2024-05-08T08:26:35.110557+0000","last_fullsized":"2024-05-08T08:26:35.110557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":346,"log_dups_size":0,"ondisk_log_size":346,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:06.557247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021349820000000001,"stat_sum":{"num_bytes":10995804,"num_objects":30,"num_object_clones":8,"num_object_copies":60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":11,"num_read":1518,"num_read_kb":13493,"num_write":326,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.e","version":"99'603","reported_seq":4598,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.096710+0000","last_change":"2024-05-08T08:26:35.092137+0000","last_active":"2024-05-08T08:26:35.096710+0000","last_peered":"2024-05-08T08:26:35.096710+0000","last_clean":"2024-05-08T08:26:35.096710+0000","last_became_active":"2024-05-08T08:25:04.615004+0000","last_became_peered":"2024-05-08T08:25:04.615004+0000","last_unstale":"2024-05-08T08:26:35.096710+0000","last_undegraded":"2024-05-08T08:26:35.096710+0000","last_fullsized":"2024-05-08T08:26:35.096710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:17.923286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020125849999999999,"stat_sum":{"num_bytes":43995266,"num_objects":26,"num_object_clones":3,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":14,"num_read":6067,"num_read_kb":47368,"num_write":611,"num_write_kb":27374,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.f","version":"99'424","reported_seq":1622,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.114763+0000","last_change":"2024-05-08T08:26:35.092489+0000","last_active":"2024-05-08T08:26:35.114763+0000","last_peered":"2024-05-08T08:26:35.114763+0000","last_clean":"2024-05-08T08:26:35.114763+0000","last_became_active":"2024-05-08T08:25:04.615942+0000","last_became_peered":"2024-05-08T08:25:04.615942+0000","last_unstale":"2024-05-08T08:26:35.114763+0000","last_undegraded":"2024-05-08T08:26:35.114763+0000","last_fullsized":"2024-05-08T08:26:35.114763+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:45:05.777369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020090849999999999,"stat_sum":{"num_bytes":5724433,"num_objects":27,"num_object_clones":6,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":11,"num_read":1820,"num_read_kb":7494,"num_write":531,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.10","version":"99'636","reported_seq":2537,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.162834+0000","last_change":"2024-05-08T08:26:35.010826+0000","last_active":"2024-05-08T08:26:35.162834+0000","last_peered":"2024-05-08T08:26:35.162834+0000","last_clean":"2024-05-08T08:26:35.162834+0000","last_became_active":"2024-05-08T08:25:04.616212+0000","last_became_peered":"2024-05-08T08:25:04.616212+0000","last_unstale":"2024-05-08T08:26:35.162834+0000","last_undegraded":"2024-05-08T08:26:35.162834+0000","last_fullsized":"2024-05-08T08:26:35.162834+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:11:35.785670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028750500000000002,"stat_sum":{"num_bytes":5284031,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":10,"num_read":3722,"num_read_kb":12455,"num_write":931,"num_write_kb":9109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.11","version":"99'425","reported_seq":1561,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.115061+0000","last_change":"2024-05-08T08:26:35.089424+0000","last_active":"2024-05-08T08:26:35.115061+0000","last_peered":"2024-05-08T08:26:35.115061+0000","last_clean":"2024-05-08T08:26:35.115061+0000","last_became_active":"2024-05-08T08:25:04.615031+0000","last_became_peered":"2024-05-08T08:25:04.615031+0000","last_unstale":"2024-05-08T08:26:35.115061+0000","last_undegraded":"2024-05-08T08:26:35.115061+0000","last_fullsized":"2024-05-08T08:26:35.115061+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:51.439260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043250999999999999,"stat_sum":{"num_bytes":23621913,"num_objects":34,"num_object_clones":9,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":15,"num_read":1689,"num_read_kb":20563,"num_write":501,"num_write_kb":8866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.12","version":"99'330","reported_seq":2777,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.112983+0000","last_change":"2024-05-08T08:26:35.014076+0000","last_active":"2024-05-08T08:26:35.112983+0000","last_peered":"2024-05-08T08:26:35.112983+0000","last_clean":"2024-05-08T08:26:35.112983+0000","last_became_active":"2024-05-08T08:25:04.609651+0000","last_became_peered":"2024-05-08T08:25:04.609651+0000","last_unstale":"2024-05-08T08:26:35.112983+0000","last_undegraded":"2024-05-08T08:26:35.112983+0000","last_fullsized":"2024-05-08T08:26:35.112983+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":330,"log_dups_size":0,"ondisk_log_size":330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:24:41.595096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067310899999999997,"stat_sum":{"num_bytes":20883517,"num_objects":40,"num_object_clones":9,"num_object_copies":80,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40,"num_whiteouts":15,"num_read":2762,"num_read_kb":31162,"num_write":380,"num_write_kb":14909,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"1b","length":"8"}]},{"pgid":"3.13","version":"99'640","reported_seq":4099,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-05-08T08:26:35.114765+0000","last_change":"2024-05-08T08:26:35.089488+0000","last_active":"2024-05-08T08:26:35.114765+0000","last_peered":"2024-05-08T08:26:35.114765+0000","last_clean":"2024-05-08T08:26:35.114765+0000","last_became_active":"2024-05-08T08:25:04.616040+0000","last_became_peered":"2024-05-08T08:25:04.616040+0000","last_unstale":"2024-05-08T08:26:35.114765+0000","last_undegraded":"2024-05-08T08:26:35.114765+0000","last_fullsized":"2024-05-08T08:26:35.114765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:20.638910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038956199999999998,"stat_sum":{"num_bytes":48287375,"num_objects":50,"num_object_clones":13,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":17,"num_read":5234,"num_read_kb":19353,"num_write":949,"num_write_kb":14291,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"1b","length":"8"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":887703258,"num_objects":1136,"num_object_clones":261,"num_object_copies":2272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1136,"num_whiteouts":492,"num_read":151721,"num_read_kb":855997,"num_write":20642,"num_write_kb":471801,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":458,"num_bytes_recovered":27477790,"num_keys_recovered":57,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45465600,"data_stored":517695248,"data_compressed":16200769,"data_compressed_allocated":39485440,"data_compressed_original":514061678,"omap_allocated":9107,"internal_metadata":0},"log_size":16523,"ondisk_log_size":16523,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39824,"kb_used_data":8640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332016,"statfs":{"total":96636764160,"available":96595984384,"internally_reserved":0,"allocated":8847360,"data_stored":89214659,"data_compressed":2910234,"data_compressed_allocated":6406144,"data_compressed_original":87885356,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37860,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333980,"statfs":{"total":96636764160,"available":96597995520,"internally_reserved":0,"allocated":2465792,"data_stored":10597240,"data_compressed":477166,"data_compressed_allocated":1060864,"data_compressed_original":9924608,"omap_allocated":1592,"internal_metadata":36174280},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40836,"kb_used_data":9652,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331004,"statfs":{"total":96636764160,"available":96594948096,"internally_reserved":0,"allocated":9883648,"data_stored":93392243,"data_compressed":2970955,"data_compressed_allocated":7401472,"data_compressed_original":92035758,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":84832,"kb_used_data":9192,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94287008,"statfs":{"total":96636764160,"available":96549896192,"internally_reserved":0,"allocated":9412608,"data_stored":89401316,"data_compressed":3002775,"data_compressed_allocated":6742016,"data_compressed_original":87951528,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38464,"kb_used_data":11444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94333376,"statfs":{"total":96636764160,"available":96597377024,"internally_reserved":0,"allocated":11718656,"data_stored":117709959,"data_compressed":3503687,"data_compressed_allocated":9080832,"data_compressed_original":116307448,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33428,"kb_used_data":2240,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338412,"statfs":{"total":96636764160,"available":96602533888,"internally_reserved":0,"allocated":2293760,"data_stored":12524406,"data_compressed":461109,"data_compressed_allocated":876544,"data_compressed_original":11823400,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":54,"seq":231928233996,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38384,"kb_used_data":5528,"kb_used_omap":13,"kb_used_meta":32818,"kb_avail":94333456,"statfs":{"total":96636764160,"available":96597458944,"internally_reserved":0,"allocated":5660672,"data_stored":46370686,"data_compressed":1232659,"data_compressed_allocated":3903488,"data_compressed_original":45473792,"omap_allocated":13550,"internal_metadata":33606418},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":79140,"kb_used_data":7656,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":94292700,"statfs":{"total":96636764160,"available":96555724800,"internally_reserved":0,"allocated":7839744,"data_stored":79141416,"data_compressed":2467251,"data_compressed_allocated":5595136,"data_compressed_original":77962231,"omap_allocated":1595,"internal_metadata":73136581},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6365184,"data_stored":76922272,"data_compressed":2399361,"data_compressed_allocated":5447680,"data_compressed_original":76379127,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4132864,"data_stored":44003751,"data_compressed":1098613,"data_compressed_allocated":3678208,"data_compressed_original":43741184,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":995328,"data_stored":11422765,"data_compressed":421122,"data_compressed_allocated":815104,"data_compressed_original":11304863,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9920512,"data_stored":114539538,"data_compressed":3352136,"data_compressed_allocated":8822784,"data_compressed_original":113901048,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":7581696,"data_stored":85506484,"data_compressed":2840977,"data_compressed_allocated":6426624,"data_compressed_original":84793846,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8212480,"data_stored":90534211,"data_compressed":2859883,"data_compressed_allocated":7176192,"data_compressed_original":89877159,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1290240,"data_stored":10045156,"data_compressed":477166,"data_compressed_allocated":1060864,"data_compressed_original":9924608,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6967296,"data_stored":84721071,"data_compressed":2751511,"data_compressed_allocated":6057984,"data_compressed_original":84139843,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:26:38.836 INFO:tasks.thrashosds.thrasher:Setting 3.3 to [2, 1] 2024-05-08T08:26:38.836 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.3', '2', '1'] 2024-05-08T08:26:38.836 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.3 2 1 2024-05-08T08:26:38.837 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:38.845+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:38.938 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:38.946+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:38.991 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:38.991 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:38.992 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:38.992 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:39.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:39.035 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:26:39.039 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:39.047+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:39.129 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:39.129 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:39.129 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:39.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:39.140 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:39.148+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:39.241 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:39.248+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:39.275 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:39.275 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:39.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 87.555697577999993, 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:39.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:39.277 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.278 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.279 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 87.286004040999998, 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.280 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.281 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.282 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.283 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.256916274999995, 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.284 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.285 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.286 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.287 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:26:39.288 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.141764578999997, 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.289 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.290 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.291 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.292 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:39.293 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:39.294 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.018118157000004, 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.295 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.296 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.297 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.298 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:39.299 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.018047010000004, 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.300 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.301 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.302 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.303 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.304 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.017701279999997, 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.305 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.306 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.307 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:39.308 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.017659991000002, 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.309 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.310 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.311 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:39.312 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.313 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.803579884000001, 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.314 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:39.315 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.316 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.317 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.511366054000007, 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.318 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.319 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:39.320 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:39.321 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.322 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:39.323 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 68.784164017999998, 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:39.324 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.325 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.326 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.327 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:39.328 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 68.565183653000005, 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.329 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.330 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.331 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.332 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.076958288, 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:39.333 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.334 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.335 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.336 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.156694133000002, 2024-05-08T08:26:39.337 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:39.338 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:39.339 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:39.340 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.341 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.008687999000003, 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.342 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.343 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.344 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.345 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.346 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.917827105000001, 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.347 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.350 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.536696861999999, 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:39.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:26:39.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:91376 3.9 e76/18 3:97e339a7:::rbd_data.1191eeab3f85.0000000000000259:head v 76'288, mlcod=76'286)", 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 28.231056985999999, 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035121610000000002, 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.355 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.356 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050141+0000", 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.95e-07 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050150+0000", 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050168+0000", 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7751e-05 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050177+0000", 2024-05-08T08:26:39.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7010000000000008e-06 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050242+0000", 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4276999999999995e-05 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050266+0000", 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4443e-05 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053607+0000", 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033412189999999999 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053653+0000", 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5287000000000002e-05 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113547 3.0 e99/57 3:012c0039:::rbd_data.11918629634e.0000000000000279:head v 99'328, mlcod=99'325)", 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:39.359 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.165516695, 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011179999999996, 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:39.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115687+0000", 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.293e-06 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115699+0000", 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-05 2024-05-08T08:26:39.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115705+0000", 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3860000000000004e-06 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115753+0000", 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7938000000000001e-05 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115770+0000", 2024-05-08T08:26:39.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6809000000000001e-05 2024-05-08T08:26:39.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119940+0000", 2024-05-08T08:26:39.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041703799999999996 2024-05-08T08:26:39.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119982+0000", 2024-05-08T08:26:39.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-05-08T08:26:39.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.365 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.365 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113549 3.9 e99/18 3:90ba2507:::rbd_data.11918629634e.000000000000027b:head v 99'310, mlcod=99'308)", 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 4.1632171150000001, 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047200790000000003, 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117981+0000", 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117988+0000", 2024-05-08T08:26:39.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0709999999999997e-06 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118003+0000", 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4701999999999999e-05 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118010+0000", 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5619999999999998e-06 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118059+0000", 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8198999999999997e-05 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118074+0000", 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5262000000000001e-05 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122655+0000", 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045809400000000004 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122700+0000", 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.369 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.370 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.370 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.370 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:39.371 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:39.372 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:39.378+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:39.472 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:39.480+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:39.522 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:39.522 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:39.522 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:39.522 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:39.531 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:39.573 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:39.580+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:39.640 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:39.640 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:39.640 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:39.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:39.693 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:39.701+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:39.760 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:39.760 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.607055115000001, 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:26:39.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:26:39.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.607045030000002, 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:26:39.765 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:26:39.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:26:39.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.769 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.607034550999998, 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:26:39.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:26:39.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:26:39.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.607017624999997, 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.776 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.778 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606996705, 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.780 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:26:39.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606958390000003, 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.784 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.786 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:26:39.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606934643999999, 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:26:39.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:26:39.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:26:39.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606911801000003, 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.792 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:39.796 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606887411999999, 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:39.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:26:39.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:26:39.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.800 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606865491999997, 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.801 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.802 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.803 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.804 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606844258000002, 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.805 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:39.806 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:26:39.807 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:26:39.808 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.606822813999997, 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.809 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.810 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:26:39.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.605838509999998, 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.813 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:26:39.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.817 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.605817309000003, 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:39.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.605805838999999, 2024-05-08T08:26:39.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:26:39.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:26:39.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.605796079000001, 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.825 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:39.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:26:39.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:26:39.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:39.829 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.605786289999998, 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:26:39.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:26:39.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.833 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.605775766000001, 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.835 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.837 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.605766086000003, 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.838 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:26:39.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 24.382504937, 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:39.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:26:39.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.845 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:39.846 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:39.846 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:39.847 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:39.847 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:39.855+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:39.947 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:39.955+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:39.963 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:39.963 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:39.963 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:39.963 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:39.973 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:40.004 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:40.021 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.3 pg_upmap mapping to [2,1] 2024-05-08T08:26:40.022 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:26:40.083 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:40.091+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:40.087 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:40.087 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:40.088 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:40.088 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:40.097 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:40.145 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:40.146 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:40.149 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:40.157+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:40.163 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:40.164 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:40.181 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:40.181 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:40.198 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:40.198 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:40.203 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:32844 3.1a 3:5c5225a1:::rbd_object_map.11915c46bf0c:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 53.053371120999998, 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0015158669999999, 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.204 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 32844 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157073+0000", 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157079+0000", 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157093+0000", 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4107999999999999e-05 2024-05-08T08:26:40.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157102+0000", 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0049999999999993e-06 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157167+0000", 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4935000000000001e-05 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157178+0000", 2024-05-08T08:26:40.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158227+0000", 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.001049232 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158355+0000", 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012803000000000001 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158587+0000", 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023214899999999999 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91168 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 595968~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.363900005000001, 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040256068999999998, 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91168 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:40.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846543+0000", 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846545+0000", 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3269999999999999e-06 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846552+0000", 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0500000000000003e-06 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846557+0000", 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886639+0000", 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040081952999999997 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886711+0000", 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2200000000000007e-05 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886798+0000", 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6879000000000006e-05 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91169 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 624640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.363878573000001, 2024-05-08T08:26:40.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040335524999999997, 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91169 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846755+0000", 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000191733 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846759+0000", 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3950000000000001e-06 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846769+0000", 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6870000000000006e-06 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846775+0000", 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4189999999999999e-06 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886812+0000", 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040036869000000003 2024-05-08T08:26:40.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886850+0000", 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8130999999999998e-05 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886899+0000", 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9291e-05 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.217 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91170 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 653312~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.363657449000002, 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040303487999999998, 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91170 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:40.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846788+0000", 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8600000000000001e-06 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846795+0000", 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2659999999999998e-06 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846800+0000", 2024-05-08T08:26:40.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886930+0000", 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040129801999999999 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887002+0000", 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2261999999999994e-05 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887088+0000", 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6491999999999998e-05 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91171 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 681984~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.363635786, 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040378757000000001, 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.222 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91171 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847633+0000", 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00082650900000000001 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847638+0000", 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7190000000000001e-06 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847652+0000", 2024-05-08T08:26:40.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3886999999999999e-05 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847659+0000", 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5480000000000004e-06 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887101+0000", 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.03944164 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887135+0000", 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4434000000000003e-05 2024-05-08T08:26:40.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887185+0000", 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0019999999999999e-05 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91176 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 825344~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.362682382999999, 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040222373999999998, 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.226 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91176 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:40.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.89e-07 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847763+0000", 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847771+0000", 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2190000000000004e-06 2024-05-08T08:26:40.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847775+0000", 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299999999999996e-06 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887825+0000", 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040049117000000002 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887897+0000", 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2799000000000006e-05 2024-05-08T08:26:40.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887982+0000", 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4901999999999995e-05 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91177 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 854016~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.362660865999999, 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040298390000000003, 2024-05-08T08:26:40.230 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91177 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847782+0000", 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.53e-07 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847784+0000", 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847791+0000", 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9e-06 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847795+0000", 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5460000000000002e-06 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887995+0000", 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040199657999999999 2024-05-08T08:26:40.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888031+0000", 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6397999999999999e-05 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888080+0000", 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8568e-05 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91178 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 882688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.234 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.362640397, 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040467768000000001, 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91178 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:40.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000001e-07 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847804+0000", 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847810+0000", 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3510000000000001e-06 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847815+0000", 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888111+0000", 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040296697999999999 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888183+0000", 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2078999999999999e-05 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888270+0000", 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6248999999999994e-05 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91179 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 911360~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.362622344999998, 2024-05-08T08:26:40.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040560845999999998, 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91179 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847822+0000", 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8080000000000001e-06 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847828+0000", 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6180000000000003e-06 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847840+0000", 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2313999999999999e-05 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888294+0000", 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040453917999999998 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888332+0000", 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7960999999999998e-05 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888381+0000", 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8807000000000002e-05 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91180 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 940032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.362595590000002, 2024-05-08T08:26:40.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040721286000000002, 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91180 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849630+0000", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017837389999999999 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849635+0000", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1330000000000001e-06 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849644+0000", 2024-05-08T08:26:40.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6239999999999997e-06 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849651+0000", 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6250000000000001e-06 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888411+0000", 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.038760599 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888483+0000", 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1863e-05 2024-05-08T08:26:40.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888568+0000", 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4703000000000005e-05 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91191 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1255424~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360539308, 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040259947999999997, 2024-05-08T08:26:40.246 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91191 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849906+0000", 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849913+0000", 2024-05-08T08:26:40.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6979999999999998e-06 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849920+0000", 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3350000000000003e-06 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890079+0000", 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040159261000000002 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890115+0000", 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5876e-05 2024-05-08T08:26:40.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890163+0000", 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8211999999999999e-05 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91192 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1284096~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360517075000001, 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040427880999999999, 2024-05-08T08:26:40.250 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91192 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:40.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849926+0000", 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849928+0000", 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849933+0000", 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3519999999999997e-06 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849938+0000", 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1780000000000002e-06 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890194+0000", 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040255304999999998 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890266+0000", 2024-05-08T08:26:40.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1979999999999999e-05 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890353+0000", 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7606000000000001e-05 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91194 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1312768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360497906999999, 2024-05-08T08:26:40.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040505864000000003, 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91194 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849945+0000", 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849947+0000", 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1490000000000001e-06 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849953+0000", 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849958+0000", 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890365+0000", 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040407605999999999 2024-05-08T08:26:40.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890401+0000", 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6044000000000003e-05 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890450+0000", 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8983000000000002e-05 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91195 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1341440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360479357999999, 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040679119999999999, 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91195 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849966+0000", 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.345e-06 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849972+0000", 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3550000000000001e-06 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849977+0000", 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1510000000000003e-06 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890481+0000", 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040503458999999999 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890551+0000", 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0686999999999996e-05 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890642+0000", 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0799999999999998e-05 2024-05-08T08:26:40.262 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91196 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1370112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360459535, 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040756042999999999, 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91196 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.263 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:40.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849985+0000", 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0619999999999999e-06 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849992+0000", 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1389999999999998e-06 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849998+0000", 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890655+0000", 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040657390000000002 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890689+0000", 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4453999999999999e-05 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890739+0000", 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9363999999999997e-05 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.266 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91197 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1398784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360439049, 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040926395999999997, 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91197 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.267 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850004+0000", 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:26:40.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850006+0000", 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.978e-06 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850012+0000", 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6680000000000004e-06 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850017+0000", 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5879999999999999e-06 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890769+0000", 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040752661000000003 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890842+0000", 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2626999999999994e-05 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890930+0000", 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7548999999999998e-05 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.270 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91198 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1427456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360404024000001, 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041001199000000002, 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91198 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.271 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850039+0000", 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2900000000000004e-07 2024-05-08T08:26:40.272 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850041+0000", 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.261e-06 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850049+0000", 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8059999999999995e-06 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850059+0000", 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8039999999999996e-06 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.273 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890955+0000", 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040895920000000002 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890990+0000", 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5760999999999999e-05 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891039+0000", 2024-05-08T08:26:40.274 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9118e-05 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91199 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1456128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360377375999999, 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041163687999999997, 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.275 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91199 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.276 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.39e-07 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850073+0000", 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1500000000000002e-06 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850084+0000", 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1827e-05 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.277 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850090+0000", 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6110000000000001e-06 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891070+0000", 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040979801000000003 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891141+0000", 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1193999999999996e-05 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.278 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891229+0000", 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7665999999999997e-05 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91200 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1484800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.360345828, 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041243876999999998, 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.279 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91200 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:40.280 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851628+0000", 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001531824 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851632+0000", 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8369999999999999e-06 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851642+0000", 2024-05-08T08:26:40.281 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4979999999999995e-06 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851651+0000", 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1859999999999998e-06 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891254+0000", 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.039602740999999997 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891292+0000", 2024-05-08T08:26:40.282 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8000999999999998e-05 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891340+0000", 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8789999999999999e-05 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91208 3.1c 3:3a43cbd7:::rbd_data.1191eeab3f85.000000000000003c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 43008] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 29.358691629999999, 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04166094, 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:40.283 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91208 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:40.284 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1799999999999995e-07 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851754+0000", 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3640000000000002e-06 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851759+0000", 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5389999999999998e-06 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.285 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851764+0000", 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.544e-06 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891864+0000", 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040100363 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892007+0000", 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000142826 2024-05-08T08:26:40.286 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892222+0000", 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021523000000000001 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893238+0000", 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001016018 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893322+0000", 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4320000000000006e-05 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.287 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893376+0000", 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3393000000000002e-05 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893412+0000", 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5824999999999997e-05 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:40.288 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:40.289 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:40.289 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:40.290 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:40.290 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:40.290 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:40.291 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:40.291 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:40.291 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:40.292 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:40.292 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:40.297+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:40.390 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:40.399+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:40.443 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:40.443 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:40.453 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:40.496 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:40.504+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:40.558 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:40.558 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:40.567 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:40.608 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:40.616+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:40.695 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:40.703+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:40.712 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:40.713 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:40.721 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:40.796 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:40.804+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:40.827 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:40.827 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:40.836 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:40.896 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:40.904+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:40.940 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:40.940 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:40.949 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:40.997 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:41.004+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:41.056 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:41.056 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:41.066 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:41.109 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:41.117+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:41.172 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:41.173 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:41.182 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:41.198 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:41.205+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:41.299 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:41.306+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:41.327 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:41.328 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:41.337 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:41.400 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:41.408+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:41.483 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:41.483 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:41.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:41.500 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:41.508+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:41.602 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:41.609+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:41.615 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:41.615 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:41.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:41.710 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:41.717+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:41.777 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:41.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:41.826 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:41.834+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:41.905 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:41.913+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:41.935 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:41.935 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:41.944 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:42.005 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:42.013+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:42.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:26:42.060 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:42.061 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:42.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:42.111 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:42.119+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:42.208 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:42.216+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:42.254 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:42.254 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:42.265 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:42.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:42.309 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:42.317+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:42.396 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:42.396 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:42.405 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:42.449 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:42.457+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:42.450 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:42.450 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:42.468 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:42.468 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:42.486 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:42.486 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:42.503 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:42.503 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:42.510 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:42.518+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:42.519 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:42.519 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:42.519 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:42.521 INFO:teuthology.orchestra.run.smithi031.stdout:osd.4: {} 2024-05-08T08:26:42.521 INFO:teuthology.orchestra.run.smithi031.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T08:26:42.528 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:42.539 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:42.539 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:42.557 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:42.557 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:42.611 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:42.618+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:42.649 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:42.649 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:42.649 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:42.649 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:42.649 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:42.658 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:42.741 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:42.749+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:42.777 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:42.777 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.091025705999996, 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.810 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.811 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.812 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:42.813 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.814 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 90.821332169000001, 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.815 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:42.816 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:42.817 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.818 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.792244402999998, 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.819 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:42.820 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.821 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.822 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.823 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 77.677092707, 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.824 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.825 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.826 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.827 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:42.828 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:42.829 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.553446285000007, 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.830 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.831 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.832 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:42.833 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:42.834 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20571 3.4 3:2796a386:::rbd_data.119174045df.0000000000000010:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.553375138000007, 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003436873, 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.835 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20571 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263150+0000", 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.836 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263151+0000", 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3900000000000005e-07 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263159+0000", 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7749999999999993e-06 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263170+0000", 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1146e-05 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.837 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263177+0000", 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7950000000000002e-06 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263432+0000", 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00025500400000000003 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263438+0000", 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7970000000000003e-06 2024-05-08T08:26:42.838 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266490+0000", 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0030514050000000001 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266587+0000", 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7411999999999995e-05 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.839 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.553029408, 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.840 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.841 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:42.842 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:42.843 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20588 3.4 3:208be467:::rbd_data.119174045df.0000000000000021:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:42.844 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.552988119000005, 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0031740639999999999, 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20588 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:42.845 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263537+0000", 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263538+0000", 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1500000000000005e-07 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.846 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263550+0000", 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1823e-05 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263558+0000", 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3389999999999993e-06 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263564+0000", 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4110000000000001e-06 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.847 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263705+0000", 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00014036499999999999 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263710+0000", 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0379999999999999e-06 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266614+0000", 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.002904004 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.848 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.266711+0000", 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7569e-05 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.338908012000005, 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.849 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.850 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.851 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.852 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.046694181999996, 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:42.853 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.854 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:42.855 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.856 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.857 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:42.858 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.319492146000002, 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:42.859 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:42.860 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.861 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.862 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:42.863 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 72.100511780999994, 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.864 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:42.865 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.866 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.867 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.868 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.612286416000003, 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.869 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.870 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:42.871 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.872 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:42.873 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.692022260999998, 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.874 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:42.875 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.876 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.877 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.878 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.544016126999999, 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:42.879 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.880 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:42.881 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:42.882 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.883 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 54.453155232999997, 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.884 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.885 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.886 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.887 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 32.072024990000003, 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.888 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.889 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:26:42.890 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.891 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:91376 3.9 e76/18 3:97e339a7:::rbd_data.1191eeab3f85.0000000000000259:head v 76'288, mlcod=76'286)", 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 31.766385113999998, 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035121610000000002, 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.892 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050140+0000", 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.893 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050141+0000", 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.95e-07 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050150+0000", 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4880000000000007e-06 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050168+0000", 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7751e-05 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.894 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050177+0000", 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7010000000000008e-06 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050242+0000", 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4276999999999995e-05 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.050266+0000", 2024-05-08T08:26:42.895 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4443e-05 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053607+0000", 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033412189999999999 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:11.053653+0000", 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5287000000000002e-05 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.896 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113547 3.0 e99/57 3:012c0039:::rbd_data.11918629634e.0000000000000279:head v 99'328, mlcod=99'325)", 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.7008448229999997, 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011179999999996, 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.897 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115687+0000", 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.293e-06 2024-05-08T08:26:42.898 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115699+0000", 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-05 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115705+0000", 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3860000000000004e-06 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115753+0000", 2024-05-08T08:26:42.899 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7938000000000001e-05 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115770+0000", 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6809000000000001e-05 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119940+0000", 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041703799999999996 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.900 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119982+0000", 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113549 3.9 e99/18 3:90ba2507:::rbd_data.11918629634e.000000000000027b:head v 99'310, mlcod=99'308)", 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.6985452429999999, 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047200790000000003, 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:42.901 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117981+0000", 2024-05-08T08:26:42.902 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117988+0000", 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0709999999999997e-06 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118003+0000", 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4701999999999999e-05 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:42.903 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118010+0000", 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5619999999999998e-06 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118059+0000", 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8198999999999997e-05 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118074+0000", 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5262000000000001e-05 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122655+0000", 2024-05-08T08:26:42.904 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045809400000000004 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122700+0000", 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:42.905 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:42.906 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:42.907 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:42.914+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:43.007 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:43.015+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:43.049 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:26:43.065 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:43.065 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:43.065 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:43.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:43.118 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:43.125+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:43.209 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:43.217+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:43.227 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:43.227 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:43.227 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:43.227 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:43.227 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:43.236 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:43.310 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:43.318+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:43.347 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:43.347 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:43.347 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:43.347 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:43.347 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.347 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.347 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193894774999997, 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:43.348 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:26:43.349 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:26:43.350 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.351 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193884689999997, 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.352 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.353 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:26:43.354 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:26:43.355 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193874211000001, 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:26:43.356 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:26:43.357 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.358 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.359 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193857285, 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.360 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.361 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.362 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.363 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.364 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193836365000003, 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:43.365 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.366 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.367 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.368 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193798049999998, 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.369 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.370 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:26:43.371 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.372 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193774304000002, 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.373 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.374 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.375 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.376 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193751460999998, 2024-05-08T08:26:43.377 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:43.378 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.379 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.380 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.381 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193727072000001, 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.382 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.383 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.384 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:26:43.385 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193705152, 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.386 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.387 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.388 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.389 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193683917999998, 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:26:43.390 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.391 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.392 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:26:43.393 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.394 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.193662474, 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.395 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.396 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.397 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:26:43.398 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.192678170000001, 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:26:43.399 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:43.400 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:26:43.401 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.402 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.192656968999998, 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.403 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.404 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:26:43.405 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:26:43.406 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.192645499000001, 2024-05-08T08:26:43.407 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:43.408 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:26:43.409 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.410 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.192635739000004, 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.412 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:43.413 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.414 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.415 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.19262595, 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.416 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.417 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.418 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.419 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.192615426000003, 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.420 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.421 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.422 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:26:43.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:43.424 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.192605745999998, 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:43.425 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.426 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.427 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:26:43.428 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.969344596999999, 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:43.429 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.430 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.431 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:43.432 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:43.433 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:43.435 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:26:43.435 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:43.442+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:43.536 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:43.543+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:43.552 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:43.552 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:43.552 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:43.552 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:43.561 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T08:26:43.636 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:43.644+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:43.707 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:43.707 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:43.707 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:43.707 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:43.707 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:43.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T08:26:43.738 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:43.745+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:32844 3.1a 3:5c5225a1:::rbd_object_map.11915c46bf0c:head [read 0~49] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e57)", 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:43.823 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 56.672432710999999, 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0015158669999999, 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 32844 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.824 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157071+0000", 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157073+0000", 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157079+0000", 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3799999999999999e-06 2024-05-08T08:26:43.825 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157093+0000", 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4107999999999999e-05 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157102+0000", 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0049999999999993e-06 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157167+0000", 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4935000000000001e-05 2024-05-08T08:26:43.826 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:47.157178+0000", 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0462000000000001e-05 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158227+0000", 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.001049232 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.827 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158355+0000", 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012803000000000001 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:48.158587+0000", 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00023214899999999999 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91168 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 595968~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:43.828 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.982961594999999, 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040256068999999998, 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91168 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.829 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846542+0000", 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846543+0000", 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1100000000000001e-07 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846545+0000", 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3269999999999999e-06 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.830 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846552+0000", 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0500000000000003e-06 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846557+0000", 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2490000000000001e-06 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886639+0000", 2024-05-08T08:26:43.831 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040081952999999997 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886711+0000", 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2200000000000007e-05 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886798+0000", 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6879000000000006e-05 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.832 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91169 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 624640~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.982940163000002, 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040335524999999997, 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91169 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:43.833 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846564+0000", 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846755+0000", 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000191733 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.834 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846759+0000", 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.3950000000000001e-06 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846769+0000", 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6870000000000006e-06 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846775+0000", 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4189999999999999e-06 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.835 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886812+0000", 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040036869000000003 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886850+0000", 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8130999999999998e-05 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886899+0000", 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9291e-05 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.836 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91170 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 653312~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.982719039000003, 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040303487999999998, 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91170 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.837 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846785+0000", 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.838 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846788+0000", 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8600000000000001e-06 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846795+0000", 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2659999999999998e-06 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846800+0000", 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2650000000000004e-06 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.839 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.886930+0000", 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040129801999999999 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887002+0000", 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2261999999999994e-05 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887088+0000", 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6491999999999998e-05 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.840 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91171 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 681984~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.982697375999997, 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040378757000000001, 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91171 2024-05-08T08:26:43.841 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.846807+0000", 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847633+0000", 2024-05-08T08:26:43.842 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00082650900000000001 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847638+0000", 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7190000000000001e-06 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847652+0000", 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3886999999999999e-05 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.843 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847659+0000", 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5480000000000004e-06 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887101+0000", 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.03944164 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887135+0000", 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4434000000000003e-05 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.844 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887185+0000", 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0019999999999999e-05 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91176 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 825344~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.981743973, 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040222373999999998, 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.845 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91176 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.846 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847760+0000", 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.89e-07 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847763+0000", 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847771+0000", 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2190000000000004e-06 2024-05-08T08:26:43.847 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847775+0000", 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7299999999999996e-06 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887825+0000", 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040049117000000002 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887897+0000", 2024-05-08T08:26:43.848 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2799000000000006e-05 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887982+0000", 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4901999999999995e-05 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91177 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 854016~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.981722456, 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040298390000000003, 2024-05-08T08:26:43.849 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91177 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.850 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847781+0000", 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847782+0000", 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.53e-07 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847784+0000", 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:43.851 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847791+0000", 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9e-06 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847795+0000", 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5460000000000002e-06 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.887995+0000", 2024-05-08T08:26:43.852 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040199657999999999 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888031+0000", 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6397999999999999e-05 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888080+0000", 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8568e-05 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.853 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91178 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 882688~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.981701987000001, 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040467768000000001, 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91178 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.854 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847802+0000", 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000001e-07 2024-05-08T08:26:43.855 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847804+0000", 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9800000000000001e-06 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847810+0000", 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3510000000000001e-06 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847815+0000", 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9189999999999998e-06 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.856 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888111+0000", 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040296697999999999 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888183+0000", 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2078999999999999e-05 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888270+0000", 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6248999999999994e-05 2024-05-08T08:26:43.857 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91179 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 911360~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.981683935, 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040560845999999998, 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.858 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91179 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.859 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847820+0000", 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847822+0000", 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.8080000000000001e-06 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847828+0000", 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6180000000000003e-06 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.860 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847840+0000", 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2313999999999999e-05 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888294+0000", 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040453917999999998 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888332+0000", 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7960999999999998e-05 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.861 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888381+0000", 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8807000000000002e-05 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91180 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 940032~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.981657179999999, 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040721286000000002, 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.862 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91180 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.847847+0000", 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.863 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849630+0000", 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017837389999999999 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849635+0000", 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1330000000000001e-06 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849644+0000", 2024-05-08T08:26:43.864 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.6239999999999997e-06 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849651+0000", 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6250000000000001e-06 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888411+0000", 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.038760599 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.865 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888483+0000", 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1863e-05 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.888568+0000", 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4703000000000005e-05 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91191 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1255424~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979600898000001, 2024-05-08T08:26:43.866 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040259947999999997, 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91191 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.867 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849903+0000", 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849906+0000", 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.868 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849913+0000", 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6979999999999998e-06 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849920+0000", 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3350000000000003e-06 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890079+0000", 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040159261000000002 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.869 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890115+0000", 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5876e-05 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890163+0000", 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8211999999999999e-05 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.870 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91192 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1284096~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979578664999998, 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040427880999999999, 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.871 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91192 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849925+0000", 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.872 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849926+0000", 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9299999999999998e-07 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849928+0000", 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.967e-06 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849933+0000", 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3519999999999997e-06 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.873 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849938+0000", 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1780000000000002e-06 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890194+0000", 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040255304999999998 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890266+0000", 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1979999999999999e-05 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.874 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890353+0000", 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7606000000000001e-05 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91194 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1312768~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979559496999997, 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040505864000000003, 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.875 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91194 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849944+0000", 2024-05-08T08:26:43.876 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849945+0000", 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5999999999999999e-07 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849947+0000", 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.1490000000000001e-06 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.877 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849953+0000", 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9510000000000003e-06 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849958+0000", 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890365+0000", 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040407605999999999 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.878 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890401+0000", 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6044000000000003e-05 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890450+0000", 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8983000000000002e-05 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91195 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1341440~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.879 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979540948, 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040679119999999999, 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91195 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849963+0000", 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2300000000000002e-07 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849966+0000", 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.345e-06 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849972+0000", 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3550000000000001e-06 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849977+0000", 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1510000000000003e-06 2024-05-08T08:26:43.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890481+0000", 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040503458999999999 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890551+0000", 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0686999999999996e-05 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890642+0000", 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0799999999999998e-05 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.883 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91196 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1370112~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979521124999998, 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040756042999999999, 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.884 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91196 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849983+0000", 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849985+0000", 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0619999999999999e-06 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849992+0000", 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1389999999999998e-06 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.886 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.849998+0000", 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890655+0000", 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040657390000000002 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890689+0000", 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4453999999999999e-05 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890739+0000", 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9363999999999997e-05 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91197 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1398784~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979500639000001, 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040926395999999997, 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.888 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91197 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850003+0000", 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850004+0000", 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2500000000000001e-07 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850006+0000", 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.978e-06 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850012+0000", 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6680000000000004e-06 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850017+0000", 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5879999999999999e-06 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890769+0000", 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040752661000000003 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890842+0000", 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2626999999999994e-05 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890930+0000", 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7548999999999998e-05 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91198 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1427456~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979465613999999, 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041001199000000002, 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.892 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91198 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850038+0000", 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850039+0000", 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2900000000000004e-07 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850041+0000", 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.261e-06 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850049+0000", 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8059999999999995e-06 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850059+0000", 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8039999999999996e-06 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890955+0000", 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040895920000000002 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.890990+0000", 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5760999999999999e-05 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891039+0000", 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9118e-05 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91199 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1456128~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979438966000004, 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041163687999999997, 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.896 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91199 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850065+0000", 2024-05-08T08:26:43.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.39e-07 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850073+0000", 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1500000000000002e-06 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850084+0000", 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1827e-05 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850090+0000", 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6110000000000001e-06 2024-05-08T08:26:43.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891070+0000", 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040979801000000003 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891141+0000", 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1193999999999996e-05 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891229+0000", 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7665999999999997e-05 2024-05-08T08:26:43.899 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91200 3.1c 3:3f0d0a6f:::rbd_data.119145616bd5.0000000000000002:22 [read 1484800~2048] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.979407418000001, 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041243876999999998, 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.900 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91200 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.850096+0000", 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851628+0000", 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001531824 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851632+0000", 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8369999999999999e-06 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851642+0000", 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4979999999999995e-06 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.902 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851651+0000", 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1859999999999998e-06 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891254+0000", 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.039602740999999997 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891292+0000", 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8000999999999998e-05 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891340+0000", 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8789999999999999e-05 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:91208 3.1c 3:3a43cbd7:::rbd_data.1191eeab3f85.000000000000003c:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,truncate 43008] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e76)", 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 32.977753219999997, 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04166094, 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:43.904 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 91208 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851751+0000", 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1799999999999995e-07 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851754+0000", 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3640000000000002e-06 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851759+0000", 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5389999999999998e-06 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:43.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.851764+0000", 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.544e-06 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.891864+0000", 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040100363 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892007+0000", 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000142826 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 5", 2024-05-08T08:26:43.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.892222+0000", 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00021523000000000001 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893238+0000", 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001016018 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893322+0000", 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4320000000000006e-05 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:43.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893376+0000", 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3393000000000002e-05 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:10.893412+0000", 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5824999999999997e-05 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:43.909 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:43.911 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:43.911 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:43.918+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:44.011 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:26:44.019+0000 7fa4c5dfc640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32420) UID: 0 2024-05-08T08:26:44.057 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:44.057 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:44.057 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:44.057 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:44.067 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:44.151 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:44.159+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:44.213 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:44.221+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:44.214 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:44.214 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:44.214 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:44.215 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:44.215 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:44.223 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:44.315 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:44.323+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:44.369 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.715329073000007, 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:44.370 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:44.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:44.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:44.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:44.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.461848052000001, 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:44.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:44.379 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.461796104000001, 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.380 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.461761248999998, 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.384 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:44.388 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.461730985000003, 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:44.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:44.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.460421619000002, 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.396 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.460400857000003, 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.397 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.460329387000002, 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.401 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.460302018999997, 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:44.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:44.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:44.409 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.458347629000002, 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.458328062, 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.414 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:44.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.418 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.458308291999998, 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:44.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.458289293999997, 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.423 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:44.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.427 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.456389657999999, 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:44.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.431 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.456369690999999, 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:44.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:44.435 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.456348118000001, 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.436 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:44.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:44.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.456224128000002, 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:44.440 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:44.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:44.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.456195332999997, 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.445 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:44.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.448 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.454277796, 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:44.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:44.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:44.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.454256342000001, 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.453 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.457 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:44.458 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:44.459 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:44.466+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:44.559 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:44.567+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:44.604 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:44.604 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:44.604 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:44.604 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:44.613 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:44.660 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:44.668+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:44.761 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:44.761 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:44.761 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:44.761 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:44.769+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:44.769 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:44.788 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:44.862 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:44.869+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:44.877 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103653346000002, 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:44.878 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.879 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.880 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:44.881 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103632628, 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.882 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.883 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:44.884 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.885 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103604691000001, 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:44.886 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:44.887 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.888 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.889 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103582928000002, 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.890 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.891 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.892 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.893 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103560350000002, 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.894 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.895 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:44.896 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.897 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103520281000002, 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.898 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:44.899 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:44.900 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:44.901 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103497973000003, 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.902 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.903 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.904 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:44.905 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103469392000001, 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:44.906 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.907 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.908 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.909 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103445139000002, 2024-05-08T08:26:44.910 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.911 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.912 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:44.913 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.103412741, 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:44.914 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.915 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:44.916 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:44.917 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.102074936000001, 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.918 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.922 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.102052917999998, 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:44.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:44.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:44.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.102010559, 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:44.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.930 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.10196904, 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.934 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.099990943999998, 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.099979650000002, 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:44.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:44.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:44.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.942 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.095891872999999, 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:44.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:44.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.946 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.095876396000001, 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:44.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:44.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.950 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.090757621000002, 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:44.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:44.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:44.954 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.090479692000002, 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:44.955 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:44.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:44.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:44.959 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:44.960 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:44.963 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:44.971+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:45.043 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:45.043 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:45.044 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 2, 3, 4, 5, 6, 7, 1] 2024-05-08T08:26:45.044 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T08:26:45.044 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:26:45.063 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:45.071+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:45.069 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:45.069 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:45.069 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:45.069 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:45.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:26:45.079 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:45.083 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T08:26:45.083 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-05-08T08:26:45.083 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:26:45.083 DEBUG:teuthology.orchestra.run.smithi149:> type ceph-objectstore-tool 2024-05-08T08:26:45.109 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:26:45.117+0000 7f144259c640 0 --2- 172.21.15.31:0/3186768169 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f14440a88c0 0x7f1444142590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-08T08:26:45.138 INFO:teuthology.orchestra.run.smithi149.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T08:26:45.139 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-05-08T08:26:45.145 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:45.146 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:45.164 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:45.171+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:45.164 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:45.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:45.182 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:45.182 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:45.201 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:45.201 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:45.230 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:45.230 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:45.231 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:45.231 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:45.231 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:45.240 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:45.264 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:45.272+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 107.795227903, 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:45.354 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.355 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:45.356 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:45.357 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:45.358 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.359 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:45.360 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.571122596000002, 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:45.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:45.364 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.571037441999998, 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:45.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.367 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:45.368 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.571009535999998, 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:45.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:45.372 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570981134999997, 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:45.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:45.376 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570896996000002, 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.380 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570816501000003, 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:45.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:45.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.384 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570783921, 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:45.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.388 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570730730000001, 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570704751000001, 2024-05-08T08:26:45.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.393 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:45.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570649418999999, 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.396 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:45.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:45.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.400 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570499386000002, 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570449727000003, 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.408 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570415158000003, 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:45.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570393150999998, 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.413 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:45.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570370177000001, 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.417 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:45.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.570340498999997, 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.421 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:45.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:45.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.569117921, 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.425 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:45.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:45.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.569107426999999, 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.429 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:45.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:45.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 43.563988590999998, 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:45.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:45.437 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:45.439 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:45.439 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:45.446+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:45.539 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:45.547+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:45.548 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:45.548 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:45.557 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:45.640 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:45.648+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:45.666 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:45.666 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:45.675 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:45.740 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:45.748+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:45.785 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:45.785 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:45.794 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:45.878 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:45.885+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:45.941 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-05-08T08:26:45.942 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:45.949+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:45.945 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:45.945 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:45.953 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:46.043 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:46.049+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:46.061 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:26:46.108 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:46.108 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:46.118 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:46.158 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:46.165+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:46.245 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:46.251+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:46.267 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:46.267 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:46.276 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:46.360 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:46.367+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:46.427 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:46.436 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:46.477 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:46.483+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:46.547 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:46.554+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:46.586 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:46.586 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:46.595 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:46.679 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:46.686+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:46.729 INFO:teuthology.orchestra.run.smithi149.stdout:3.f 2024-05-08T08:26:46.729 INFO:teuthology.orchestra.run.smithi149.stdout:3.e 2024-05-08T08:26:46.729 INFO:teuthology.orchestra.run.smithi149.stdout:3.9 2024-05-08T08:26:46.729 INFO:teuthology.orchestra.run.smithi149.stdout:3.1c 2024-05-08T08:26:46.729 INFO:teuthology.orchestra.run.smithi149.stdout:3.1a 2024-05-08T08:26:46.729 INFO:teuthology.orchestra.run.smithi149.stdout:3.1 2024-05-08T08:26:46.729 INFO:teuthology.orchestra.run.smithi149.stdout:3.a 2024-05-08T08:26:46.730 INFO:teuthology.orchestra.run.smithi149.stdout:3.1b 2024-05-08T08:26:46.730 INFO:teuthology.orchestra.run.smithi149.stdout:3.14 2024-05-08T08:26:46.730 INFO:teuthology.orchestra.run.smithi149.stdout:3.13 2024-05-08T08:26:46.730 INFO:teuthology.orchestra.run.smithi149.stdout:3.11 2024-05-08T08:26:46.730 INFO:teuthology.orchestra.run.smithi149.stdout:3.1d 2024-05-08T08:26:46.730 INFO:teuthology.orchestra.run.smithi149.stdout:1.5 2024-05-08T08:26:46.745 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:46.745 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:46.748 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:46.755+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:46.754 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:46.849 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:46.857+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:46.859 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:46.859 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:46.860 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:46.860 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:46.868 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:46.952 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:46.960+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:47.015 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:47.015 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:47.015 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:47.015 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:47.015 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:47.024 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:47.065 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:47.073+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:47.151 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:47.159+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:47.185 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:47.185 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 77.530994644000003, 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.186 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:47.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:47.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:47.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.277513622999997, 2024-05-08T08:26:47.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.192 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.195 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.277461674999998, 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:47.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.199 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.277426820000002, 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:47.203 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.277396555999999, 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.204 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:47.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:47.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.276087189999998, 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:47.208 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.212 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.276066428, 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:47.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.275994957999998, 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.217 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:47.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:47.221 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.27596759, 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:47.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:47.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.225 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.274013199999999, 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:47.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.229 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.273993633000003, 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:47.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.233 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.273973863000002, 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:47.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:47.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.273954865, 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.238 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:47.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.242 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.272055229000003, 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:47.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:47.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:47.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.246 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.272035262000003, 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:47.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:47.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:47.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.272013688999998, 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:47.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.254 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.271889698999999, 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.255 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.256 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.257 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.271860904, 2024-05-08T08:26:47.258 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.259 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:47.260 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:47.261 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.269943367000003, 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.262 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.263 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:47.264 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:47.265 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.269921912999997, 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.266 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.267 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.268 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.269 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.270 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:47.271 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:47.272 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 3.1 --file /var/log/ceph/exp.3.1.4 2024-05-08T08:26:47.273 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:47.279+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:47.371 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:47.379+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:47.420 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:47.420 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:47.421 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:47.421 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:47.429 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:47.472 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:47.480+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:47.574 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:47.581+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:47.575 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:47.575 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:47.575 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:47.575 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:47.575 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:47.584 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:47.675 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:47.682+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:47.697 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.697 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.923121688999998, 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:47.698 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:47.699 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:47.700 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:47.701 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.923100970999997, 2024-05-08T08:26:47.702 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.703 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.704 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.705 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.706 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.923073033999998, 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:47.707 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:47.708 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:47.709 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:47.710 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.923051270999999, 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.711 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.712 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.713 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.714 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.923028692999999, 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.715 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.716 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.717 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:47.718 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.719 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.922988623999998, 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:47.720 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.721 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:47.722 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:47.723 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:47.724 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.922966316, 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.725 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:47.726 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.727 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.728 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.922937734999998, 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.729 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:47.730 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.731 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.732 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.922913481999998, 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.733 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:47.734 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.735 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.736 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:47.737 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.922881083999997, 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.738 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.739 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.740 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.741 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.921543278999998, 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.742 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.743 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.744 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.745 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.921521261000002, 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.746 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:47.747 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.748 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.749 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.921478901999997, 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.750 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.751 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.752 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:47.753 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.921437382999997, 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.754 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:47.755 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:47.756 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:47.757 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.919459287000002, 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.758 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.759 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.760 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.761 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.919447992999999, 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.762 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.763 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:47.764 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:47.765 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.915360216000003, 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.766 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.767 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:47.768 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:47.769 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.915344738999998, 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.770 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.771 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.772 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.773 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.910225963999999, 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.774 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:47.775 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:47.776 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:47.777 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 45.909948034999999, 2024-05-08T08:26:47.778 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.779 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:47.780 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:47.781 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:47.782 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:47.783 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:47.784 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:47.784 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:47.791+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:47.885 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:47.891+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:47.930 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:47.930 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:47.930 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:47.930 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:47.938 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:47.986 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:47.993+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:48.043 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:48.043 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:48.043 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:48.043 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:48.043 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:48.052 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:48.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:26:48.123 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:48.130+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:48.169 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 110.610338337, 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:48.170 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.171 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.172 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.173 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:48.174 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.175 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.38623303, 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.176 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.177 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.178 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.179 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.386147876000003, 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.180 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.181 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.182 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.183 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.386119970000003, 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.184 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.185 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:48.186 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.187 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.386091569000001, 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.188 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.189 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.190 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.191 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.386007429999999, 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:48.192 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:48.193 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.194 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.195 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385926935000001, 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.196 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.197 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:48.198 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.199 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:48.200 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385894354999998, 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.201 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:48.202 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.203 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385841163999999, 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:48.204 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.205 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.206 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:48.207 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385815184999998, 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.208 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.209 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:48.210 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.211 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385759853000003, 2024-05-08T08:26:48.212 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.213 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.214 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.215 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.216 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385609819999999, 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:48.217 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.218 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.219 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.220 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385560161000001, 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.221 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:48.222 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:48.223 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.224 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385525592, 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.225 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.226 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.227 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.228 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385503585000002, 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.229 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.230 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.231 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:48.232 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385480610999998, 2024-05-08T08:26:48.233 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.234 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.235 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.236 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.237 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.385450933000001, 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.238 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:48.239 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.240 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.241 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.384228354999998, 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.242 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.243 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:48.244 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.245 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.384217861000003, 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:48.246 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.247 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.248 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 46.379099025000002, 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:48.249 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.250 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.251 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.252 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:48.253 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:48.255 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:48.255 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:48.261+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:48.355 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:48.363+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:48.365 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:48.365 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:48.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:48.405 INFO:teuthology.orchestra.run.smithi149.stderr:Exporting 3.1 info 3.1( v 108'1203 (0'0,108'1203] local-lis/les=18/19 n=42 ec=18/18 lis/c=18/18 les/c/f=19/19/0 sis=18) 2024-05-08T08:26:48.419 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:80329143:::rbd_data.1191948f0501.0000000000000078:head# 2024-05-08T08:26:48.421 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:8092a9da:::rbd_data.119177d1b31a.0000000000000039:head# 2024-05-08T08:26:48.421 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:809679b4:::rbd_data.1191f7f15880.0000000000000029:head# 2024-05-08T08:26:48.421 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:80ccb53f:::rbd_data.1191f60358f6.00000000000000e5:14# 2024-05-08T08:26:48.421 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:80ccb53f:::rbd_data.1191f60358f6.00000000000000e5:head# 2024-05-08T08:26:48.421 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:81aaa67d:::rbd_data.119137f21733.000000000000000f:13# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:81aaa67d:::rbd_data.119137f21733.000000000000000f:head# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:81ae232f:::rbd_header.1191f7f15880:head# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:81bf8398:::rbd_data.119137f21733.0000000000000360:13# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:81bf8398:::rbd_data.119137f21733.0000000000000360:head# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:81c2ee89:::rbd_data.1191f7f15880.0000000000000063:head# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:825dbdd8:::rbd_data.119116cc7dd0.000000000000038b:4# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:825dbdd8:::rbd_data.119116cc7dd0.000000000000038b:head# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:828b5196:::rbd_data.1191f7f15880.0000000000000058:head# 2024-05-08T08:26:48.423 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:828b8991:::rbd_data.1191c87c6f01.0000000000000000:head# 2024-05-08T08:26:48.426 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:829a956c:::rbd_data.119174045df.0000000000000006:head# 2024-05-08T08:26:48.428 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:8319318f:::rbd_object_map.1191ea0f1b1f:head# 2024-05-08T08:26:48.428 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:83561f1e:::rbd_data.11916d7efae6.000000000000001a:head# 2024-05-08T08:26:48.428 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:837fa337:::rbd_data.11911e381319.000000000000005a:12# 2024-05-08T08:26:48.428 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:837fa337:::rbd_data.11911e381319.000000000000005a:head# 2024-05-08T08:26:48.429 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:848e12fd:::rbd_data.1191f60358f6.0000000000000002:14# 2024-05-08T08:26:48.430 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:848e12fd:::rbd_data.1191f60358f6.0000000000000002:head# 2024-05-08T08:26:48.430 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:84a3e745:::rbd_data.119116cc7dd0.00000000000001e6:4# 2024-05-08T08:26:48.430 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:84a3e745:::rbd_data.119116cc7dd0.00000000000001e6:head# 2024-05-08T08:26:48.430 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:8511d695:::rbd_data.1191f7f15880.0000000000000081:head# 2024-05-08T08:26:48.430 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:857b23f8:::rbd_data.1191948f0501.000000000000000a:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:85b27194:::rbd_data.1191f7f15880.0000000000000000:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:85c4154b:::rbd_data.1191f7f15880.0000000000000067:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:862e50b2:::rbd_data.1191f7f15880.0000000000000050:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:86b2513a:::rbd_data.1191f60358f6.0000000000000052:14# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:86b2513a:::rbd_data.1191f60358f6.0000000000000052:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:86b31e72:::rbd_data.11916d7efae6.000000000000001e:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:8709b772:::rbd_data.119116cc7dd0.0000000000000206:4# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:8709b772:::rbd_data.119116cc7dd0.0000000000000206:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:870fab5c:::rbd_data.1191f7f15880.0000000000000056:head# 2024-05-08T08:26:48.432 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:8712925e:::rbd_data.1191f7f15880.0000000000000071:head# 2024-05-08T08:26:48.433 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:87164b57:::rbd_data.1191f60358f6.0000000000000051:14# 2024-05-08T08:26:48.433 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:87164b57:::rbd_data.1191f60358f6.0000000000000051:head# 2024-05-08T08:26:48.433 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:87405074:::rbd_data.1191f7f15880.000000000000002a:head# 2024-05-08T08:26:48.433 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:87be67d2:::rbd_data.11911e381319.0000000000000014:12# 2024-05-08T08:26:48.433 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:87be67d2:::rbd_data.11911e381319.0000000000000014:head# 2024-05-08T08:26:48.433 INFO:teuthology.orchestra.run.smithi149.stderr:Read #3:87f2aa45:::rbd_data.1191c87c6f01.0000000000000006:head# 2024-05-08T08:26:48.434 INFO:teuthology.orchestra.run.smithi149.stderr:Export successful 2024-05-08T08:26:48.456 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:48.464+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:48.483 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:48.484 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:48.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:48.557 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:48.563+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:48.602 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:48.602 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:48.611 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:48.657 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:48.665+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:48.758 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:48.766+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:48.761 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:48.761 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:48.770 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:48.859 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:48.866+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:48.881 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:48.882 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:48.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:48.956 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 3.1 2024-05-08T08:26:48.960 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:48.966+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:48.969 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:26:49.013 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:49.062 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:49.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:49.064 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:49.071+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:49.164 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:49.172+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:49.214 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:49.214 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:49.223 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:49.266 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:49.273+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:49.332 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:49.332 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:49.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:49.383 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:49.389+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:49.467 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:49.475+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:49.491 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:49.491 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:49.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:49.567 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:49.574+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:49.610 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:49.610 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:49.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:49.702 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:49.709+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:49.768 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:49.769 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:49.769 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:49.776+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:49.777 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:49.870 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:49.876+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:49.927 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:49.927 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:49.936 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:49.977 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:49.984+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:50.046 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:50.046 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:50.046 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:50.046 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:50.054 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:50.098 INFO:teuthology.orchestra.run.smithi149.stdout: marking collection for removal 2024-05-08T08:26:50.098 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:50.106+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:50.099 INFO:teuthology.orchestra.run.smithi149.stdout:setting '_remove' omap key 2024-05-08T08:26:50.099 INFO:teuthology.orchestra.run.smithi149.stdout:finish_remove_pgs 3.1_head removing 3.1 2024-05-08T08:26:50.103 INFO:teuthology.orchestra.run.smithi149.stdout:Remove successful 2024-05-08T08:26:50.111 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:26:50.129 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:50.129 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:50.147 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:50.147 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:50.165 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:50.165 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:50.165 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:50.165 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:50.169 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:26:50.169 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:26:50.173 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:50.181+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:50.174 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:50.274 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:50.281+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:50.319 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:50.319 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:50.328 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:50.374 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:50.382+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:50.434 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:50.434 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:50.444 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:50.475 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:50.482+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:50.576 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:50.583+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:50.591 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:50.591 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:50.600 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:50.618 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.1.4 2024-05-08T08:26:50.702 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:50.710+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:50.765 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:50.766 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:50.775 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:50.816 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:50.823+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:50.878 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:50.886+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:50.922 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:50.922 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:50.932 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:50.979 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:50.986+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:50.990 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:26:51.080 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:51.086+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:51.111 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:51.111 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:51.127 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:51.181 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:51.187+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:51.250 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:51.250 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:51.259 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:51.282 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:51.289+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:51.371 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:51.371 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:51.380 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:51.382 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:51.390+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:51.482 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:51.490+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:51.490 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:51.490 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:51.499 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:51.583 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:51.590+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:51.613 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:51.613 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:51.623 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:51.708 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:51.715+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:51.761 INFO:teuthology.orchestra.run.smithi149.stdout:get_pg_num_history pg_num_history pg_num_history(e108 pg_nums {1={14=8},2={15=1},3={18=32}} deleted_pools ) 2024-05-08T08:26:51.761 INFO:teuthology.orchestra.run.smithi149.stdout:Importing pgid 3.1 2024-05-08T08:26:51.762 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:80329143:::rbd_data.1191948f0501.0000000000000078:head# 2024-05-08T08:26:51.762 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 0=[]:{} 2024-05-08T08:26:51.763 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:8092a9da:::rbd_data.119177d1b31a.0000000000000039:head# 2024-05-08T08:26:51.763 INFO:teuthology.orchestra.run.smithi149.stdout:snapset b=[]:{} 2024-05-08T08:26:51.763 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:809679b4:::rbd_data.1191f7f15880.0000000000000029:head# 2024-05-08T08:26:51.763 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.764 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:80ccb53f:::rbd_data.1191f60358f6.00000000000000e5:14# 2024-05-08T08:26:51.765 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:80ccb53f:::rbd_data.1191f60358f6.00000000000000e5:head# 2024-05-08T08:26:51.765 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 14=[]:{14=[14]} 2024-05-08T08:26:51.766 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:81aaa67d:::rbd_data.119137f21733.000000000000000f:13# 2024-05-08T08:26:51.767 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:81aaa67d:::rbd_data.119137f21733.000000000000000f:head# 2024-05-08T08:26:51.767 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 13=[]:{13=[13]} 2024-05-08T08:26:51.767 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:81ae232f:::rbd_header.1191f7f15880:head# 2024-05-08T08:26:51.768 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 0=[]:{} 2024-05-08T08:26:51.768 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:81bf8398:::rbd_data.119137f21733.0000000000000360:13# 2024-05-08T08:26:51.769 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:81bf8398:::rbd_data.119137f21733.0000000000000360:head# 2024-05-08T08:26:51.769 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 13=[]:{13=[13]} 2024-05-08T08:26:51.769 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:81c2ee89:::rbd_data.1191f7f15880.0000000000000063:head# 2024-05-08T08:26:51.769 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.770 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:825dbdd8:::rbd_data.119116cc7dd0.000000000000038b:4# 2024-05-08T08:26:51.771 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:825dbdd8:::rbd_data.119116cc7dd0.000000000000038b:head# 2024-05-08T08:26:51.771 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 4=[]:{4=[4]} 2024-05-08T08:26:51.771 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:828b5196:::rbd_data.1191f7f15880.0000000000000058:head# 2024-05-08T08:26:51.771 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.771 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:828b8991:::rbd_data.1191c87c6f01.0000000000000000:head# 2024-05-08T08:26:51.772 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 0=[]:{} 2024-05-08T08:26:51.774 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:829a956c:::rbd_data.119174045df.0000000000000006:head# 2024-05-08T08:26:51.774 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 0=[]:{} 2024-05-08T08:26:51.774 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:51.774 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:51.775 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:8319318f:::rbd_object_map.1191ea0f1b1f:head# 2024-05-08T08:26:51.775 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 0=[]:{} 2024-05-08T08:26:51.775 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:83561f1e:::rbd_data.11916d7efae6.000000000000001a:head# 2024-05-08T08:26:51.775 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 26=[]:{} 2024-05-08T08:26:51.776 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:837fa337:::rbd_data.11911e381319.000000000000005a:12# 2024-05-08T08:26:51.777 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:837fa337:::rbd_data.11911e381319.000000000000005a:head# 2024-05-08T08:26:51.777 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 12=[]:{12=[12]} 2024-05-08T08:26:51.777 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:848e12fd:::rbd_data.1191f60358f6.0000000000000002:14# 2024-05-08T08:26:51.778 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:848e12fd:::rbd_data.1191f60358f6.0000000000000002:head# 2024-05-08T08:26:51.778 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 14=[]:{14=[14]} 2024-05-08T08:26:51.778 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:84a3e745:::rbd_data.119116cc7dd0.00000000000001e6:4# 2024-05-08T08:26:51.779 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:84a3e745:::rbd_data.119116cc7dd0.00000000000001e6:head# 2024-05-08T08:26:51.779 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 4=[]:{4=[4]} 2024-05-08T08:26:51.779 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:8511d695:::rbd_data.1191f7f15880.0000000000000081:head# 2024-05-08T08:26:51.779 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.780 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:857b23f8:::rbd_data.1191948f0501.000000000000000a:head# 2024-05-08T08:26:51.780 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 0=[]:{} 2024-05-08T08:26:51.780 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:85b27194:::rbd_data.1191f7f15880.0000000000000000:head# 2024-05-08T08:26:51.781 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.781 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:85c4154b:::rbd_data.1191f7f15880.0000000000000067:head# 2024-05-08T08:26:51.781 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.781 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:862e50b2:::rbd_data.1191f7f15880.0000000000000050:head# 2024-05-08T08:26:51.782 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.782 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:86b2513a:::rbd_data.1191f60358f6.0000000000000052:14# 2024-05-08T08:26:51.783 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:51.784 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:86b2513a:::rbd_data.1191f60358f6.0000000000000052:head# 2024-05-08T08:26:51.784 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 14=[]:{14=[14]} 2024-05-08T08:26:51.784 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:86b31e72:::rbd_data.11916d7efae6.000000000000001e:head# 2024-05-08T08:26:51.784 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 26=[]:{} 2024-05-08T08:26:51.785 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:8709b772:::rbd_data.119116cc7dd0.0000000000000206:4# 2024-05-08T08:26:51.785 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:8709b772:::rbd_data.119116cc7dd0.0000000000000206:head# 2024-05-08T08:26:51.785 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 4=[]:{4=[4]} 2024-05-08T08:26:51.786 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:870fab5c:::rbd_data.1191f7f15880.0000000000000056:head# 2024-05-08T08:26:51.786 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.786 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:8712925e:::rbd_data.1191f7f15880.0000000000000071:head# 2024-05-08T08:26:51.786 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.786 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:87164b57:::rbd_data.1191f60358f6.0000000000000051:14# 2024-05-08T08:26:51.787 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:87164b57:::rbd_data.1191f60358f6.0000000000000051:head# 2024-05-08T08:26:51.787 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 14=[]:{14=[14]} 2024-05-08T08:26:51.787 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:87405074:::rbd_data.1191f7f15880.000000000000002a:head# 2024-05-08T08:26:51.787 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 6=[]:{} 2024-05-08T08:26:51.788 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:87be67d2:::rbd_data.11911e381319.0000000000000014:12# 2024-05-08T08:26:51.790 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:87be67d2:::rbd_data.11911e381319.0000000000000014:head# 2024-05-08T08:26:51.790 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 12=[]:{12=[12]} 2024-05-08T08:26:51.790 INFO:teuthology.orchestra.run.smithi149.stdout:Write #3:87f2aa45:::rbd_data.1191c87c6f01.0000000000000006:head# 2024-05-08T08:26:51.791 INFO:teuthology.orchestra.run.smithi149.stdout:snapset 0=[]:{} 2024-05-08T08:26:51.793 INFO:teuthology.orchestra.run.smithi149.stdout:write_pg epoch 108 info 3.1( v 108'1203 (0'0,108'1203] local-lis/les=18/19 n=42 ec=18/18 lis/c=18/18 les/c/f=19/19/0 sis=18) 2024-05-08T08:26:51.801 INFO:teuthology.orchestra.run.smithi149.stdout:Import successful 2024-05-08T08:26:51.825 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:51.832+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:51.885 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:51.893+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:51.933 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:51.934 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:51.942 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:51.973 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:26:51.986 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:51.994+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:52.056 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:52.057 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:52.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:52.109 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:52.116+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:52.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-05-08T08:26:52.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:52.176 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:52.176 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:52.186 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:52.188 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:52.196+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:52.289 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:52.296+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:52.310 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f /var/log/ceph/exp.3.1.4 2024-05-08T08:26:52.311 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:52.311 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:52.330 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:52.330 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:52.347 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:52.347 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:52.347 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:52.348 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:52.348 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:52.357 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:52.365 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:52.366 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:52.366 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:26:52.383 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:52.383 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:52.401 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:52.401 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:52.419 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:52.419 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:52.419 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:52.427+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:52.423 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:26:52.423 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.759022464, 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.478 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:52.479 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:52.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 100.489328927, 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.482 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.483 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.484 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:52.485 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14413 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 87.460241160999999, 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035859109999999998, 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.486 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14413 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024281+0000", 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024283+0000", 2024-05-08T08:26:52.487 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024292+0000", 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1109999999999997e-06 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024316+0000", 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3682000000000001e-05 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024327+0000", 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1236000000000001e-05 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.488 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024393+0000", 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5779999999999997e-05 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.024461+0000", 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8231000000000005e-05 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027761+0000", 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0032998490000000001 2024-05-08T08:26:52.489 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027796+0000", 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.4959999999999997e-05 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.027867+0000", 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0887000000000001e-05 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.490 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 87.345089465000001, 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.491 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:52.492 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.493 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.494 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.495 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.221443042999994, 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.496 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:52.497 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:52.498 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.499 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.221026166000001, 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:52.500 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.501 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.502 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:52.503 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.504 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.006904770000006, 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.505 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.506 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.507 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 82.714690939999997, 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.508 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.509 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:52.510 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:52.511 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:52.512 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 81.987488904000003, 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.513 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:52.514 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.515 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:52.516 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.517 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 81.768508538999995, 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.518 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.519 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.520 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.521 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 74.280283174000004, 2024-05-08T08:26:52.522 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.523 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:52.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.360019019000006, 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.526 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:52.530 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.212012884999993, 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:52.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:52.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.534 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 64.121151991000005, 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:52.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 41.740021747999997, 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113547 3.0 e99/57 3:012c0039:::rbd_data.11918629634e.0000000000000279:head v 99'328, mlcod=99'325)", 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.368841581000002, 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011179999999996, 2024-05-08T08:26:52.543 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115687+0000", 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.293e-06 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115699+0000", 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-05 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115705+0000", 2024-05-08T08:26:52.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3860000000000004e-06 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115753+0000", 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7938000000000001e-05 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115770+0000", 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6809000000000001e-05 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119940+0000", 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041703799999999996 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119982+0000", 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113549 3.9 e99/18 3:90ba2507:::rbd_data.11918629634e.000000000000027b:head v 99'310, mlcod=99'308)", 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.366542000999999, 2024-05-08T08:26:52.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047200790000000003, 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:52.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117981+0000", 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117988+0000", 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0709999999999997e-06 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118003+0000", 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4701999999999999e-05 2024-05-08T08:26:52.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118010+0000", 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5619999999999998e-06 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118059+0000", 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8198999999999997e-05 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118074+0000", 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5262000000000001e-05 2024-05-08T08:26:52.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122655+0000", 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045809400000000004 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122700+0000", 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:114608 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:26:52.551 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 9.3697839609999996, 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95824956900000002, 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114608 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114740+0000", 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5069999999999999e-06 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114747+0000", 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114762+0000", 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5464999999999998e-05 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114770+0000", 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8599999999999993e-06 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114842+0000", 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1748999999999994e-05 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114856+0000", 2024-05-08T08:26:52.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3976e-05 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071293+0000", 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95643727300000003 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071421+0000", 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127718 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071573+0000", 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015250100000000001 2024-05-08T08:26:52.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072356+0000", 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000783141 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072850+0000", 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049327699999999997 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072933+0000", 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2993999999999999e-05 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072988+0000", 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5170000000000002e-05 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:543 3.0 e109/57 3:04449ea6:::rbd_data.1191f60358f6.000000000000000c:14 v 109'338, mlcod=102'335)", 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.3882098669999996, 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037001569999999999, 2024-05-08T08:26:52.557 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096313+0000", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096319+0000", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.251e-06 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096332+0000", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3107e-05 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096341+0000", 2024-05-08T08:26:52.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096414+0000", 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3201000000000006e-05 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096437+0000", 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2891000000000002e-05 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.099956+0000", 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035193490000000002 2024-05-08T08:26:52.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.100013+0000", 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6258000000000002e-05 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:544 3.0 e109/57 3:02813c57:::rbd_data.1191f60358f6.0000000000000058:14 v 109'340, mlcod=102'335)", 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 7.3874393549999997, 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057648059999999999, 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:52.561 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097084+0000", 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.562 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097092+0000", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5189999999999992e-06 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097105+0000", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2685e-05 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097112+0000", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0049999999999998e-06 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097176+0000", 2024-05-08T08:26:52.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4166999999999997e-05 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097203+0000", 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000001e-05 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102791+0000", 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055883590000000002 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102848+0000", 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6789999999999997e-05 2024-05-08T08:26:52.564 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:52.565 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:52.565 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:52.566 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:52.566 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:52.573+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:52.667 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:52.673+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:52.717 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:52.717 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:52.717 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:52.717 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:52.725 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:52.768 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:52.775+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:52.868 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:52.876+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:52.875 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:52.875 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:52.875 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:52.884 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:52.970 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:52.977+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:53.001 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:53.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.848067282000002, 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.002 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:26:53.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.848057197000003, 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.006 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.848046717999999, 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.010 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:53.014 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.848029791999998, 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:53.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.018 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.848008872000001, 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:26:53.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:26:53.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.022 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.847970556999996, 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.847946811, 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.030 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.847923968000003, 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:26:53.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:26:53.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.847899579, 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:26:53.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:26:53.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.039 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.847877658999998, 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:53.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:26:53.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:26:53.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.847856425000003, 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.847834980999998, 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:26:53.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.846850676999999, 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:26:53.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.846829476000003, 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.846818005999999, 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.846808246000002, 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:26:53.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.846798456999998, 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.068 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.071 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.846787933000002, 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:26:53.075 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 60.846778252999997, 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.076 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.077 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:26:53.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.623517104000001, 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.080 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.082 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:26:53.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:53.084 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:53.085 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:53.086 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:53.093+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:53.186 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:53.194+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:53.190 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:53.191 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:53.191 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:53.191 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:53.200 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:53.287 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:53.294+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:53.346 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:53.346 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:53.346 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:53.346 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:53.346 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:53.355 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:53.388 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:53.395+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:53.463 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 83.808750386, 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:26:53.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:26:53.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:26:53.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:26:53.468 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.555269365000001, 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.555217417000001, 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:26:53.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.555182561999999, 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.481 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.555152298000003, 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.482 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.553842932000002, 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.489 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.553822169999997, 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:26:53.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:26:53.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:26:53.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:26:53.493 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.553750700000002, 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.494 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:26:53.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.553723331999997, 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:26:53.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:26:53.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.501 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.551768942000002, 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.502 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.551749375, 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.506 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.551729604999998, 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:53.514 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.551710606999997, 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.518 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.549810970999999, 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:53.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:26:53.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.549791003999999, 2024-05-08T08:26:53.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:26:53.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.549769431000001, 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.526 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.549645441000003, 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:26:53.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:26:53.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.549616645999997, 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:26:53.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:26:53.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.537 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.547699109, 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:26:53.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:26:53.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:26:53.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.541 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.547677655000001, 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.542 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:26:53.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:26:53.546 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.546 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.546 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.546 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.546 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.546 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:53.547 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:53.548 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:53.554+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:53.648 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:53.655+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:53.653 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:53.653 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:53.653 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:53.653 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:53.662 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:53.748 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:53.755+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:53.807 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:53.807 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:53.807 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:53.808 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:53.808 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:53.816 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:53.849 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:53.856+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:53.922 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149235253999997, 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:53.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:53.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:53.927 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149214536000002, 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149186598999997, 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.935 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149164835999997, 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:53.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:53.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149142257999998, 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.940 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:53.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:53.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149102188999997, 2024-05-08T08:26:53.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.948 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149079880999999, 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.949 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:53.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.953 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149051299999996, 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.954 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.957 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:53.958 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.149027046999997, 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.961 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:53.962 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.148994649000002, 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:53.964 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.147656843999997, 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.967 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:53.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.147634826000001, 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.972 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:53.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:53.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:53.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.147592467000003, 2024-05-08T08:26:53.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:53.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:53.981 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.147550948000003, 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.982 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.145572852000001, 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:53.986 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:53.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.988 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:53.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.990 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.145561557999997, 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.991 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.141473781000002, 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.995 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:53.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:53.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:53.999 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.141458303999997, 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.000 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.004 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.136339528999997, 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:54.005 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:54.006 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:54.007 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.008 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.136061599999998, 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.009 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.010 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.011 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.013 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.014 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:54.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:26:54.015 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:54.022+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:54.016 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:54.116 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:54.123+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:54.129 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:54.129 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:54.129 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:54.129 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:54.138 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:54.216 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:54.224+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:54.244 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:54.244 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:54.244 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:54.244 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:54.244 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:54.253 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:54.317 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:54.325+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 116.800882569, 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.360 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.361 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:54.362 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:54.363 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.364 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.365 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.366 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576777262, 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.367 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.368 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.369 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.370 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576692108000003, 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.371 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.372 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.373 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:54.374 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576664202000003, 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.375 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.376 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.377 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:54.378 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576635801000002, 2024-05-08T08:26:54.379 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.380 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:54.381 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:54.382 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.383 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576551662, 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:54.384 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.385 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.386 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.387 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.388 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576471167000001, 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.389 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.390 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.391 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.392 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576438586999998, 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.393 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.394 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:54.395 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.396 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576385395999999, 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.397 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.398 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:54.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.401 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576359416999999, 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.402 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.406 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576304084999997, 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.407 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.411 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576154052, 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.412 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:54.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576104393000001, 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.416 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:54.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.420 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576069824000001, 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.421 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:54.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:54.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576047817000003, 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:54.425 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:54.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.429 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.576024842999999, 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.430 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:54.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:54.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:54.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:54.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:54.434 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.575995165000002, 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.574772586999998, 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.439 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.574762092999997, 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.443 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.447 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 52.569643257000003, 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:54.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:54.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:54.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:54.452 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.452 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.452 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.452 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:54.452 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:54.452 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:54.452 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:54.454 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:54.454 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:54.461+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:54.554 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:54.561+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:54.568 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:54.568 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:54.568 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:54.577 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:54.583 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:54.584 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:54.602 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:54.602 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:54.619 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:54.619 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:54.637 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:54.637 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:54.637 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:26:54.654 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:54.662+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:54.655 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:54.655 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:54.672 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:54.673 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:54.690 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:54.690 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:54.693 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:26:54.694 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:26:54.694 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:54.694 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:54.703 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:54.787 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:54.795+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:54.853 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:54.853 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:54.856 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:54.863+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:54.862 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:54.956 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:54.964+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:54.994 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:26:55.016 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:55.017 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:55.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:55.058 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:55.065+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:55.158 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:55.166+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:55.181 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:55.182 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:55.191 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:55.274 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:55.282+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:55.340 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:55.340 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:55.349 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:55.391 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:55.398+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:55.461 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:55.469+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:55.499 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:55.499 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:55.508 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:55.563 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:55.570+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:55.658 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:55.658 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:55.664 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:55.672+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:55.667 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:55.765 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:55.772+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:55.777 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:55.777 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:55.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:55.865 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:55.873+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:55.937 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:55.937 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:55.946 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:55.990 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:55.998+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:56.065 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:56.065 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:56.066 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:56.074+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:56.076 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:56.167 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:56.175+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:56.228 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:56.228 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:56.237 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:56.268 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:56.276+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:56.348 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:56.348 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:56.357 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:56.402 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:56.410+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:56.464 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:56.464 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:56.470 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:56.478+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:56.474 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:56.571 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:56.579+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:56.620 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:56.620 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:56.629 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:56.672 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:56.680+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:56.695 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:26:56.734 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:56.734 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:56.743 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:26:56.788 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:56.796+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:56.825 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:56.826 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:56.843 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:56.843 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:56.850 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:56.850 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:56.860 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:26:56.861 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:56.861 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:56.877 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:56.878 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:26:56.895 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:56.896 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:56.902 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:56.910+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:56.913 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:56.914 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:56.931 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:56.931 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:56.934 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:26:56.935 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:26:56.975 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:56.983+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:57.010 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236716051000002, 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.011 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.012 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.013 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.014 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236695333, 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.015 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.016 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.017 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:26:57.018 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236667396000001, 2024-05-08T08:26:57.019 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.020 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.021 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:26:57.022 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.023 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236645633000002, 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.024 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:57.025 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.026 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.027 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.028 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236623055000003, 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.029 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:26:57.030 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:26:57.031 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:26:57.032 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236582986000002, 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.033 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:26:57.034 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.035 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.036 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.037 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236560677999996, 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.038 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:26:57.039 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.040 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.041 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:57.042 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236532097000001, 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:57.043 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.044 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:26:57.045 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:26:57.046 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236507844000002, 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.047 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.048 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.049 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.050 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.051 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.236475446, 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.052 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.053 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:26:57.054 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:26:57.055 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.235137641000001, 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:26:57.056 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.057 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.058 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.059 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:26:57.060 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.235115622999999, 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.061 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.062 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.063 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.064 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.065 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.235073264, 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.066 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.067 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.068 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.069 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.070 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.235031745000001, 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.071 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.072 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:26:57.073 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.074 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.075 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.233053648999999, 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.076 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:26:57.077 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:26:57.078 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.079 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.233042355000002, 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.080 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.081 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:26:57.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.228954578, 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.085 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.086 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:26:57.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:26:57.089 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.228939101000002, 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.090 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.094 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.223820326000002, 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:26:57.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.223542397000003, 2024-05-08T08:26:57.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:26:57.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.104 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:57.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:26:57.106 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:57.113+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:57.106 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:26:57.206 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:57.213+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:57.213 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:57.213 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:57.213 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:26:57.214 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:57.223 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:26:57.307 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:57.314+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:57.329 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:57.329 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:26:57.329 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:26:57.329 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:26:57.330 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:57.339 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:26:57.378 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [4] live_osds: [0, 2, 3, 5, 6, 7, 1] 2024-05-08T08:26:57.379 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:26:57.379 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T08:26:57.379 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 6 2024-05-08T08:26:57.423 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:57.431+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:57.488 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 119.929127104, 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:57.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:26:57.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:26:57.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.493 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:26:57.494 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.705021797000001, 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.495 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:26:57.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704936643000003, 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.499 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:26:57.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:26:57.503 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704908736999997, 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.504 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:26:57.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704880336000002, 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.508 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.512 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704796197, 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:57.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:26:57.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704715702000001, 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:26:57.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.520 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704683121999999, 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:26:57.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704629930999999, 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.525 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.526 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:57.529 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704603951999999, 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:26:57.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:26:57.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.533 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704548619999997, 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.534 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.537 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704398587, 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.538 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.539 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.540 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.541 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704348928000002, 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.542 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.543 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.544 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:26:57.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:26:57.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704314359000001, 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.548 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:26:57.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704292352000003, 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.552 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:26:57.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704269377999999, 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.557 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:26:57.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:26:57.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:57.561 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.704239700000002, 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.703017121999999, 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.566 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:26:57.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:57.570 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.703006627999997, 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:57.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:26:57.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.574 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 55.697887792000003, 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.575 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:26:57.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:26:57.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:26:57.579 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:26:57.581 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:26:57.582 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:57.588+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:57.682 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:57.689+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:57.691 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:57.692 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:57.692 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:57.692 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-05-08T08:26:57.699+0000 7f2e11ed7640 -1 mon.a@0(leader).osd e121 definitely_dead 0 2024-05-08T08:26:57.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:26:57.783 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:57.791+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:57.850 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:57.850 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:57.850 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:57.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:26:57.883 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:26:57.891+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:26:57.970 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.816822369999997, 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:57.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:57.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.816801269999999, 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:57.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:26:57.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:57.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 65.816780612000002, 2024-05-08T08:26:57.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:57.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108422 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982776735000002, 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032000842000000002, 2024-05-08T08:26:57.983 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108422 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994307+0000", 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:57.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994314+0000", 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994320+0000", 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7760000000000003e-06 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026109+0000", 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031789339 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:57.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026194+0000", 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4841999999999998e-05 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026301+0000", 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106667 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108428 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982749126000002, 2024-05-08T08:26:57.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032197602999999998, 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108428 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:57.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994328+0000", 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994330+0000", 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5689999999999999e-06 2024-05-08T08:26:57.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994338+0000", 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5830000000000001e-06 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994343+0000", 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026333+0000", 2024-05-08T08:26:57.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031990022 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026419+0000", 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6278000000000003e-05 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026525+0000", 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010569 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108429 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:57.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982725846000001, 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032394017999999997, 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108429 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994353+0000", 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3829999999999999e-06 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994360+0000", 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6850000000000004e-06 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994365+0000", 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2680000000000002e-06 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026559+0000", 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032193927999999997 2024-05-08T08:26:57.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026647+0000", 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7882999999999999e-05 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026745+0000", 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7460999999999994e-05 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108430 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3211264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982703878000002, 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032603935000000001, 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108430 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:57.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:57.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994375+0000", 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3989999999999998e-06 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994383+0000", 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2869999999999999e-06 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994387+0000", 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026777+0000", 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032389793 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026865+0000", 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7763999999999995e-05 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026976+0000", 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111488 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:57.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108431 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3244032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982681470999999, 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032796189000000003, 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108431 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994398+0000", 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994404+0000", 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3969999999999999e-06 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994410+0000", 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027010+0000", 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032600259999999999 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027094+0000", 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128999999999994e-05 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027191+0000", 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7089000000000005e-05 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108444 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982659601000002, 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032973852999999997, 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108444 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:58.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994419+0000", 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2900000000000001e-06 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994426+0000", 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5080000000000002e-06 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994431+0000", 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0200000000000002e-06 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027223+0000", 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032791703999999998 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027303+0000", 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0068999999999995e-05 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027391+0000", 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7899000000000007e-05 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108445 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982638577000003, 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033150011, 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108445 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994441+0000", 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994448+0000", 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9929999999999998e-06 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994452+0000", 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8199999999999996e-06 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027420+0000", 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032967989000000003 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027500+0000", 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9332000000000005e-05 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027588+0000", 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8080999999999999e-05 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108446 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982616317999998, 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033416003, 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108446 2024-05-08T08:26:58.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994461+0000", 2024-05-08T08:26:58.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994463+0000", 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994470+0000", 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4489999999999996e-06 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994475+0000", 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027699+0000", 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033223821000000001 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027779+0000", 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0411e-05 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027876+0000", 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6855999999999996e-05 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108447 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.982594274, 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033598876, 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108447 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:26:58.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994483+0000", 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994485+0000", 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994492+0000", 2024-05-08T08:26:58.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994497+0000", 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3569999999999997e-06 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027912+0000", 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033415331999999999 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027991+0000", 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8825999999999994e-05 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.028081+0000", 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9858000000000006e-05 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108675 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.977598354999998, 2024-05-08T08:26:58.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032085805000000002, 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108675 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.022 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999480+0000", 2024-05-08T08:26:58.023 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.793e-06 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999486+0000", 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999490+0000", 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.464e-06 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031407+0000", 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031916959000000002 2024-05-08T08:26:58.024 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031488+0000", 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0567000000000003e-05 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031564+0000", 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6114999999999999e-05 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108676 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.025 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.977580857, 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032251646000000002, 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108676 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.026 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999498+0000", 2024-05-08T08:26:58.027 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999503+0000", 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999509+0000", 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8270000000000003e-06 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.028 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031594+0000", 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032084797999999998 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031672+0000", 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8765999999999998e-05 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031747+0000", 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4881000000000004e-05 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.029 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108677 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.977562100999997, 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032405850999999999, 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108677 2024-05-08T08:26:58.030 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.031 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999515+0000", 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.79e-07 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999516+0000", 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999522+0000", 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2460000000000003e-06 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.032 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999526+0000", 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.673e-06 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031777+0000", 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032250681000000003 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031851+0000", 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3694000000000003e-05 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.033 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031920+0000", 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9449999999999994e-05 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108678 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.977544795999997, 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032573421999999998, 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.034 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108678 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.035 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0800000000000004e-07 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999538+0000", 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999545+0000", 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.036 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999549+0000", 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031947+0000", 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032397903999999998 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032029+0000", 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1985999999999997e-05 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.037 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032105+0000", 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6032999999999995e-05 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108703 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6832128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.976217452, 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03217747, 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.038 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108703 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.039 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000861+0000", 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8199999999999999e-06 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000866+0000", 2024-05-08T08:26:58.040 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0880000000000002e-06 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000871+0000", 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510000000000004e-06 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032860+0000", 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031989640999999999 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.041 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032962+0000", 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010197200000000001 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033036+0000", 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4299000000000001e-05 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108704 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6864896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.975868347999999, 2024-05-08T08:26:58.042 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032008086999999998, 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108704 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.043 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001209+0000", 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5199999999999996e-07 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001213+0000", 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8770000000000003e-06 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.044 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001224+0000", 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1342000000000001e-05 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001231+0000", 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033067+0000", 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031835544 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033149+0000", 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1719999999999997e-05 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033216+0000", 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7645999999999996e-05 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108705 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:58.046 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.975832992000001, 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032150629999999999, 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108705 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001244+0000", 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001247+0000", 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001254+0000", 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7190000000000001e-06 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001260+0000", 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5099999999999998e-06 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033245+0000", 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031984929000000002 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033322+0000", 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7435999999999995e-05 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033394+0000", 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1740000000000001e-05 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.050 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108706 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6930432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.975805299000001, 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032385262999999997, 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108706 2024-05-08T08:26:58.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001272+0000", 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:26:58.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001274+0000", 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001281+0000", 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0840000000000003e-06 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001287+0000", 2024-05-08T08:26:58.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033490+0000", 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032203455999999998 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033573+0000", 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2860000000000003e-05 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033656+0000", 2024-05-08T08:26:58.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3380999999999993e-05 2024-05-08T08:26:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.055 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.055 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:58.056 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:26:58.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:26:58.057 INFO:teuthology.orchestra.run.smithi031.stderr:marked out osd.6. 2024-05-08T08:26:58.058 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:58.063+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:58.157 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:58.165+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:58.207 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:58.207 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:58.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:26:58.258 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:58.266+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:58.359 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:26:58.366+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:26:58.365 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:58.365 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:58.365 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:58.374 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:26:58.459 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:58.467+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:58.524 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:58.524 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:58.524 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:58.524 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69714 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.710076708000003, 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046246284999999998, 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69714 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.525 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820801+0000", 2024-05-08T08:26:58.526 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820803+0000", 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820809+0000", 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7410000000000003e-06 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.527 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820815+0000", 2024-05-08T08:26:58.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6200000000000004e-06 2024-05-08T08:26:58.528 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.528 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.528 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.528 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866947+0000", 2024-05-08T08:26:58.528 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046131923999999998 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866993+0000", 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867046+0000", 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3052e-05 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.529 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69715 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3162112~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.710054552999999, 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046369965999999999, 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69715 2024-05-08T08:26:58.530 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820823+0000", 2024-05-08T08:26:58.531 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820825+0000", 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.655e-06 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820832+0000", 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0849999999999999e-06 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820838+0000", 2024-05-08T08:26:58.532 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867075+0000", 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046236823000000003 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867133+0000", 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.533 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867192+0000", 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9475999999999998e-05 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69716 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3391488~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.710031053000002, 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046544169000000003, 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.534 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69716 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.535 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820848+0000", 2024-05-08T08:26:58.536 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820855+0000", 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140000000000002e-06 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820861+0000", 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867234+0000", 2024-05-08T08:26:58.537 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046372732 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867299+0000", 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5485000000000001e-05 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867390+0000", 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969000000000003e-05 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.538 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69717 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3620864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.710009057999997, 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046712535999999999, 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.539 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69717 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:58.540 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820870+0000", 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1320000000000001e-06 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.541 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820879+0000", 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820892+0000", 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3356000000000001e-05 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867422+0000", 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046529981999999998 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.542 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867485+0000", 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2614999999999996e-05 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867580+0000", 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5359000000000006e-05 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.543 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69718 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709976197000003, 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046863712000000002, 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69718 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:58.544 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820904+0000", 2024-05-08T08:26:58.545 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820911+0000", 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820917+0000", 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.546 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867609+0000", 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046691734999999998 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867681+0000", 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1931e-05 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867764+0000", 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3356000000000006e-05 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.547 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69719 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4079616~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709952485999999, 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047037906999999997, 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69719 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.548 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.549 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820925+0000", 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820927+0000", 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.550 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820934+0000", 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.764e-06 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820939+0000", 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.553e-06 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867796+0000", 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046856496999999997 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.551 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867868+0000", 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2163999999999996e-05 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867962+0000", 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4337000000000002e-05 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69720 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4308992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.552 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.70993034, 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047192222999999998, 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69720 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.553 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.554 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820949+0000", 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820956+0000", 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820963+0000", 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.555 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867988+0000", 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047025083000000002 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868053+0000", 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4871999999999998e-05 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868139+0000", 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5626999999999999e-05 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.556 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69721 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4538368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709905939000002, 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047345685999999998, 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69721 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.557 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:26:58.558 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820974+0000", 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820982+0000", 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8370000000000003e-06 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820988+0000", 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.559 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868167+0000", 2024-05-08T08:26:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047179062000000001 2024-05-08T08:26:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.560 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868232+0000", 2024-05-08T08:26:58.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4889999999999997e-05 2024-05-08T08:26:58.561 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.561 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.561 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.561 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868317+0000", 2024-05-08T08:26:58.561 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4244000000000002e-05 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69722 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4767744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709881258000003, 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047510657999999997, 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.562 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69722 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.563 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821004+0000", 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821011+0000", 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6510000000000001e-06 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.564 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821019+0000", 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868347+0000", 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047327859 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868419+0000", 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2725999999999995e-05 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.565 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868506+0000", 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6793999999999996e-05 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69723 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4997120~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709849740999999, 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047669162000000001, 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.566 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69723 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.567 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821028+0000", 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821030+0000", 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.568 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821037+0000", 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821043+0000", 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868536+0000", 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047493103000000002 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.569 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868611+0000", 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4647999999999994e-05 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868696+0000", 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5476999999999995e-05 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69724 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5226496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.570 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709826462000002, 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819591000000002, 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69724 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:58.571 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.572 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.572 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:26:58.572 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.572 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.572 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821051+0000", 2024-05-08T08:26:58.573 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-05-08T08:26:58.573 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.573 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.573 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.573 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821053+0000", 2024-05-08T08:26:58.574 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-05-08T08:26:58.574 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.574 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.574 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.574 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821060+0000", 2024-05-08T08:26:58.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.097e-06 2024-05-08T08:26:58.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.575 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.575 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.575 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821066+0000", 2024-05-08T08:26:58.575 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:26:58.575 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.576 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.576 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.576 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868722+0000", 2024-05-08T08:26:58.576 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047656233999999999 2024-05-08T08:26:58.576 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.577 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.577 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868795+0000", 2024-05-08T08:26:58.577 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2621999999999997e-05 2024-05-08T08:26:58.577 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.577 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.578 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.578 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868870+0000", 2024-05-08T08:26:58.578 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5352999999999995e-05 2024-05-08T08:26:58.578 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.578 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.579 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.579 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.579 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.579 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69725 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5455872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.579 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:58.579 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709804169999998, 2024-05-08T08:26:58.579 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047973017999999999, 2024-05-08T08:26:58.580 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.580 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.580 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.580 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69725 2024-05-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:58.581 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.582 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.582 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:58.582 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.582 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.582 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:26:58.583 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-05-08T08:26:58.583 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.583 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.583 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.583 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821075+0000", 2024-05-08T08:26:58.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.356e-06 2024-05-08T08:26:58.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.584 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.584 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.584 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821082+0000", 2024-05-08T08:26:58.584 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:26:58.584 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821088+0000", 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868907+0000", 2024-05-08T08:26:58.585 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819382000000001 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868979+0000", 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1922999999999996e-05 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.586 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869046+0000", 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6724999999999996e-05 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.587 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69726 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5685248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709764747000001, 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048091651999999999, 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.588 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.589 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.589 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69726 2024-05-08T08:26:58.589 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.589 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.589 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.589 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.589 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:58.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.590 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.590 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.590 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:26:58.590 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.590 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.591 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.591 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821113+0000", 2024-05-08T08:26:58.591 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-05-08T08:26:58.591 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.591 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.592 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821115+0000", 2024-05-08T08:26:58.592 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7889999999999999e-06 2024-05-08T08:26:58.592 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.592 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.592 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821124+0000", 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.191e-06 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821130+0000", 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:26:58.593 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869065+0000", 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047935187999999997 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.594 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869129+0000", 2024-05-08T08:26:58.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4356999999999994e-05 2024-05-08T08:26:58.595 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.595 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.595 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.595 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869204+0000", 2024-05-08T08:26:58.595 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399000000000004e-05 2024-05-08T08:26:58.595 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.596 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.596 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.596 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.596 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.596 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69727 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5914624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.596 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:58.596 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709738639999998, 2024-05-08T08:26:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048288931, 2024-05-08T08:26:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.597 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69727 2024-05-08T08:26:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.598 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:26:58.599 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821139+0000", 2024-05-08T08:26:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-05-08T08:26:58.600 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821141+0000", 2024-05-08T08:26:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-05-08T08:26:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.601 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821148+0000", 2024-05-08T08:26:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:26:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.602 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821153+0000", 2024-05-08T08:26:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-05-08T08:26:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869256+0000", 2024-05-08T08:26:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048102180000000001 2024-05-08T08:26:58.603 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869343+0000", 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7828e-05 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869427+0000", 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3666999999999997e-05 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.604 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69728 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6144000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709716192000002, 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048487729, 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.605 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69728 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.606 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821164+0000", 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821171+0000", 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.607 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821176+0000", 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869489+0000", 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048312213999999999 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869566+0000", 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7657999999999994e-05 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.608 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869648+0000", 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2194000000000006e-05 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69729 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6373376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709693270000002, 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048680186, 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.609 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69729 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.610 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821186+0000", 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.611 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821192+0000", 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821197+0000", 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869699+0000", 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048501191999999999 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.612 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869782+0000", 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2973000000000007e-05 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869864+0000", 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2241000000000001e-05 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69730 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6602752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.613 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709673072000001, 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048837093999999998, 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.614 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69730 2024-05-08T08:26:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:58.615 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.616 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.616 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.616 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.616 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:58.616 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:26:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:26:58.617 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821206+0000", 2024-05-08T08:26:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:26:58.618 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821212+0000", 2024-05-08T08:26:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:26:58.619 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821218+0000", 2024-05-08T08:26:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-05-08T08:26:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.620 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869928+0000", 2024-05-08T08:26:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048710003000000002 2024-05-08T08:26:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.621 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869995+0000", 2024-05-08T08:26:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7125000000000005e-05 2024-05-08T08:26:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.622 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870041+0000", 2024-05-08T08:26:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6134999999999998e-05 2024-05-08T08:26:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.623 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69731 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6832128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709651999999998, 2024-05-08T08:26:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048956353000000001, 2024-05-08T08:26:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.624 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69731 2024-05-08T08:26:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.625 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:58.626 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:26:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:26:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.627 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821228+0000", 2024-05-08T08:26:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-05-08T08:26:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.628 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821234+0000", 2024-05-08T08:26:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:26:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821239+0000", 2024-05-08T08:26:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.485e-06 2024-05-08T08:26:58.629 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870066+0000", 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048826621000000001 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.630 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870133+0000", 2024-05-08T08:26:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:26:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870181+0000", 2024-05-08T08:26:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8553000000000001e-05 2024-05-08T08:26:58.631 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69732 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7061504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:58.632 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709613111000003, 2024-05-08T08:26:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049133148000000001, 2024-05-08T08:26:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.633 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69732 2024-05-08T08:26:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.634 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.635 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:26:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-05-08T08:26:58.636 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821267+0000", 2024-05-08T08:26:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-05-08T08:26:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.637 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821286+0000", 2024-05-08T08:26:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9122000000000001e-05 2024-05-08T08:26:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821292+0000", 2024-05-08T08:26:58.638 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:26:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870231+0000", 2024-05-08T08:26:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048938161000000001 2024-05-08T08:26:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.639 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870316+0000", 2024-05-08T08:26:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5143999999999997e-05 2024-05-08T08:26:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.640 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870397+0000", 2024-05-08T08:26:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1118000000000005e-05 2024-05-08T08:26:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.641 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69733 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7290880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 56.709575786000002, 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049266982000000001, 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:58.642 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69733 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.643 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821302+0000", 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821304+0000", 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821311+0000", 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729999999999998e-06 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.644 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821317+0000", 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2660000000000001e-06 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870426+0000", 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049109685 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870497+0000", 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0860999999999998e-05 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:58.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870568+0000", 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0984999999999999e-05 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:58.646 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:58.648 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:26:58.648 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:58.655+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:58.749 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:58.757+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:58.798 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:58.798 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:26:58.798 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:58.807 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:26:58.849 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:26:58.857+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:26:58.916 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:26:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:26:58.916 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:26:58.916 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:58.925 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:26:58.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:26:58.951 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:58.958+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:59.044 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:26:59.044 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:26:59.044 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.325427777, 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.045 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:26:59.046 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:26:59.047 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.048 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.05573424000001, 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.049 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.050 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.051 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.052 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.911494778000005, 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.053 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.054 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.055 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.056 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.057 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.787848355999998, 2024-05-08T08:26:59.058 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.059 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.060 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:26:59.061 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:26:59.062 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.787431479000006, 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:26:59.063 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.064 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.065 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.066 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.067 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.573310082999996, 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.068 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:26:59.069 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.070 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:26:59.071 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.281096253000001, 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:26:59.072 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.073 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.074 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.075 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:26:59.076 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.553894217000007, 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:26:59.077 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:26:59.078 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.079 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:26:59.080 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.081 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 88.334913852, 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.082 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.083 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:26:59.084 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:26:59.085 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:26:59.086 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.846688486999994, 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:26:59.087 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:26:59.088 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:26:59.089 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.090 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.926424331999996, 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.091 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.092 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.093 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:59.094 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.095 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.778418197999997, 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.099 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.687557303999995, 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.103 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 48.306427061000001, 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:26:59.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113547 3.0 e99/57 3:012c0039:::rbd_data.11918629634e.0000000000000279:head v 99'328, mlcod=99'325)", 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.935246893999999, 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011179999999996, 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:26:59.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115687+0000", 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.293e-06 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115699+0000", 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-05 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115705+0000", 2024-05-08T08:26:59.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3860000000000004e-06 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115753+0000", 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7938000000000001e-05 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115770+0000", 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6809000000000001e-05 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119940+0000", 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041703799999999996 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119982+0000", 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113549 3.9 e99/18 3:90ba2507:::rbd_data.11918629634e.000000000000027b:head v 99'310, mlcod=99'308)", 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 23.932947314, 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047200790000000003, 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117981+0000", 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117988+0000", 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0709999999999997e-06 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118003+0000", 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4701999999999999e-05 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118010+0000", 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5619999999999998e-06 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118059+0000", 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8198999999999997e-05 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118074+0000", 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5262000000000001e-05 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122655+0000", 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045809400000000004 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122700+0000", 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:114608 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 15.936189274, 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95824956900000002, 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114608 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114740+0000", 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5069999999999999e-06 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114747+0000", 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114762+0000", 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5464999999999998e-05 2024-05-08T08:26:59.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114770+0000", 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8599999999999993e-06 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114842+0000", 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1748999999999994e-05 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114856+0000", 2024-05-08T08:26:59.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3976e-05 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071293+0000", 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95643727300000003 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071421+0000", 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127718 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:59.121 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071573+0000", 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015250100000000001 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072356+0000", 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000783141 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072850+0000", 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049327699999999997 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072933+0000", 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2993999999999999e-05 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072988+0000", 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5170000000000002e-05 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:543 3.0 e109/57 3:04449ea6:::rbd_data.1191f60358f6.000000000000000c:14 v 109'338, mlcod=102'335)", 2024-05-08T08:26:59.123 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.954615179999999, 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037001569999999999, 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:26:59.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096313+0000", 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096319+0000", 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.251e-06 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096332+0000", 2024-05-08T08:26:59.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3107e-05 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096341+0000", 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096414+0000", 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3201000000000006e-05 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096437+0000", 2024-05-08T08:26:59.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2891000000000002e-05 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.099956+0000", 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035193490000000002 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.100013+0000", 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6258000000000002e-05 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:544 3.0 e109/57 3:02813c57:::rbd_data.1191f60358f6.0000000000000058:14 v 109'340, mlcod=102'335)", 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 13.953844668, 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057648059999999999, 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097084+0000", 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097092+0000", 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5189999999999992e-06 2024-05-08T08:26:59.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097105+0000", 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2685e-05 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097112+0000", 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0049999999999998e-06 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097176+0000", 2024-05-08T08:26:59.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4166999999999997e-05 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097203+0000", 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000001e-05 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102791+0000", 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055883590000000002 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102848+0000", 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6789999999999997e-05 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:115393 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 5.013729025, 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036588620000000001, 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:26:59.132 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115393 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:26:59.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037200+0000", 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.86e-06 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037209+0000", 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7090000000000008e-06 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037229+0000", 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0049000000000001e-05 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037238+0000", 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6640000000000004e-06 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037305+0000", 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7300999999999998e-05 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037311+0000", 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1029999999999998e-06 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039469+0000", 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021582839999999999 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039564+0000", 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4547000000000001e-05 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039753+0000", 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018868800000000001 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040757+0000", 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010045869999999999 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040784+0000", 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6840999999999999e-05 2024-05-08T08:26:59.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040824+0000", 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9956000000000002e-05 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040857+0000", 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3272999999999998e-05 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:26:59.138 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:26:59.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:26:59.140 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:26:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:26:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:26:59.141 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:26:59.141 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:26:59.142 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:26:59.142 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:59.147+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:59.155 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:26:59.156 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:26:59.156 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:26:59.174 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:26:59.174 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:26:59.193 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:26:59.193 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:26:59.211 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:26:59.212 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:26:59.214 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:26:59.215 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:26:59.256 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:59.264+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:59.265 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:59.265 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:59.275 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:26:59.342 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:26:59.349+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:26:59.424 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:59.424 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:59.433 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:26:59.476 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:59.484+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:59.543 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:59.543 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:59.544 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:26:59.551+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:26:59.552 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:26:59.644 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:59.651+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:59.670 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:59.670 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:59.680 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:26:59.745 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:59.753+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:26:59.786 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:59.786 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:59.796 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:26:59.846 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:26:59.854+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:26:59.902 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:26:59.902 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:26:59.911 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:26:59.947 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:26:59.955+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:00.038 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:00.038 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:00.047 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:00.055+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:00.050 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:27:00.059 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:27:00.148 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:00.156+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:00.179 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:00.179 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:00.189 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:27:00.249 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:00.257+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:00.304 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:00.304 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:00.314 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:27:00.350 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:00.357+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:00.451 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:00.458+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:00.465 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:00.465 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:00.474 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:27:00.552 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:00.559+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:00.624 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:00.624 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:00.633 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:27:00.653 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:00.660+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:00.745 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:00.745 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:00.754 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:27:00.755 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:00.762+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:00.854 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:00.862+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:00.865 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:00.865 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:00.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:27:00.957 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:00.965+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:01.025 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:01.026 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:01.033 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:27:01.039 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:27:01.056 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:01.064+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:01.156 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:01.164+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:01.200 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:01.200 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:01.209 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:27:01.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:27:01.296 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:01.304+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:01.320 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:01.320 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:01.330 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:27:01.372 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:01.379+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:01.381 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:27:01.381 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:27:01.399 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:27:01.399 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:27:01.418 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:27:01.418 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:27:01.435 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:27:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:27:01.436 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:27:01.454 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:27:01.454 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:27:01.459 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:01.466+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:01.471 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:27:01.472 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:27:01.483 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:01.483 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:01.483 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:01.489 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:27:01.489 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:27:01.492 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:27:01.493 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:27:01.493 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:27:01.560 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:01.567+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:01.643 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:01.643 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69714 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829609048999998, 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046246284999999998, 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69714 2024-05-08T08:27:01.644 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820801+0000", 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-05-08T08:27:01.645 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820803+0000", 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820809+0000", 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7410000000000003e-06 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820815+0000", 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6200000000000004e-06 2024-05-08T08:27:01.646 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866947+0000", 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046131923999999998 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866993+0000", 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867046+0000", 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3052e-05 2024-05-08T08:27:01.647 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69715 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3162112~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829586894000002, 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046369965999999999, 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69715 2024-05-08T08:27:01.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820823+0000", 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:27:01.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820825+0000", 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.655e-06 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820832+0000", 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0849999999999999e-06 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820838+0000", 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-05-08T08:27:01.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867075+0000", 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046236823000000003 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867133+0000", 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867192+0000", 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9475999999999998e-05 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69716 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3391488~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829563393999997, 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046544169000000003, 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69716 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820848+0000", 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820855+0000", 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140000000000002e-06 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820861+0000", 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867234+0000", 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046372732 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867299+0000", 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5485000000000001e-05 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867390+0000", 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969000000000003e-05 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69717 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3620864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829541399, 2024-05-08T08:27:01.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046712535999999999, 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69717 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820870+0000", 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1320000000000001e-06 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820879+0000", 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820892+0000", 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3356000000000001e-05 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867422+0000", 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046529981999999998 2024-05-08T08:27:01.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867485+0000", 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2614999999999996e-05 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867580+0000", 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5359000000000006e-05 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.660 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69718 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829508537999999, 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046863712000000002, 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69718 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:01.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820904+0000", 2024-05-08T08:27:01.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820911+0000", 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820917+0000", 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867609+0000", 2024-05-08T08:27:01.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046691734999999998 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867681+0000", 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1931e-05 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867764+0000", 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3356000000000006e-05 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.664 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69719 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4079616~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829484827000002, 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047037906999999997, 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69719 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:27:01.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820925+0000", 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820927+0000", 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820934+0000", 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.764e-06 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820939+0000", 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.553e-06 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867796+0000", 2024-05-08T08:27:01.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046856496999999997 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867868+0000", 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2163999999999996e-05 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867962+0000", 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4337000000000002e-05 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69720 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4308992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829462681000003, 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047192222999999998, 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69720 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820949+0000", 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820956+0000", 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820963+0000", 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867988+0000", 2024-05-08T08:27:01.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047025083000000002 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868053+0000", 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4871999999999998e-05 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868139+0000", 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5626999999999999e-05 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69721 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4538368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829438279999998, 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047345685999999998, 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69721 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:01.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820974+0000", 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-05-08T08:27:01.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820982+0000", 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8370000000000003e-06 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820988+0000", 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868167+0000", 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047179062000000001 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868232+0000", 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4889999999999997e-05 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868317+0000", 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4244000000000002e-05 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69722 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4767744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829413598999999, 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047510657999999997, 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69722 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821004+0000", 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821011+0000", 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6510000000000001e-06 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821019+0000", 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:27:01.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868347+0000", 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047327859 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868419+0000", 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2725999999999995e-05 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868506+0000", 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6793999999999996e-05 2024-05-08T08:27:01.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69723 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4997120~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829382082000002, 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047669162000000001, 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69723 2024-05-08T08:27:01.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:27:01.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821028+0000", 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821030+0000", 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821037+0000", 2024-05-08T08:27:01.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821043+0000", 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868536+0000", 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047493103000000002 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868611+0000", 2024-05-08T08:27:01.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4647999999999994e-05 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868696+0000", 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5476999999999995e-05 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69724 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5226496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829358802999998, 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819591000000002, 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69724 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821051+0000", 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821053+0000", 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821060+0000", 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.097e-06 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821066+0000", 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868722+0000", 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047656233999999999 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868795+0000", 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2621999999999997e-05 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868870+0000", 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5352999999999995e-05 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69725 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5455872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829336511000001, 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047973017999999999, 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69725 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821075+0000", 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.356e-06 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821082+0000", 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821088+0000", 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868907+0000", 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819382000000001 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868979+0000", 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1922999999999996e-05 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869046+0000", 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6724999999999996e-05 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69726 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5685248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829297087999997, 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048091651999999999, 2024-05-08T08:27:01.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69726 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821113+0000", 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821115+0000", 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7889999999999999e-06 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821124+0000", 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.191e-06 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821130+0000", 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869065+0000", 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047935187999999997 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869129+0000", 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4356999999999994e-05 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869204+0000", 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399000000000004e-05 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69727 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5914624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829270981000001, 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048288931, 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69727 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821139+0000", 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821141+0000", 2024-05-08T08:27:01.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821148+0000", 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821153+0000", 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869256+0000", 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048102180000000001 2024-05-08T08:27:01.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869343+0000", 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7828e-05 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869427+0000", 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3666999999999997e-05 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69728 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6144000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829248532999998, 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048487729, 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69728 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821164+0000", 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821171+0000", 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-05-08T08:27:01.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821176+0000", 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869489+0000", 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048312213999999999 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869566+0000", 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7657999999999994e-05 2024-05-08T08:27:01.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869648+0000", 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2194000000000006e-05 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69729 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6373376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829225610999998, 2024-05-08T08:27:01.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048680186, 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69729 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821186+0000", 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821192+0000", 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821197+0000", 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869699+0000", 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048501191999999999 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869782+0000", 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2973000000000007e-05 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869864+0000", 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2241000000000001e-05 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69730 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6602752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829205412999997, 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048837093999999998, 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69730 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821206+0000", 2024-05-08T08:27:01.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821212+0000", 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821218+0000", 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869928+0000", 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048710003000000002 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869995+0000", 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7125000000000005e-05 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870041+0000", 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6134999999999998e-05 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69731 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6832128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829184341000001, 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048956353000000001, 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69731 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821228+0000", 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821234+0000", 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821239+0000", 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.485e-06 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870066+0000", 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048826621000000001 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870133+0000", 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870181+0000", 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8553000000000001e-05 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69732 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7061504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829145451999999, 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049133148000000001, 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69732 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821267+0000", 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821286+0000", 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9122000000000001e-05 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821292+0000", 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870231+0000", 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048938161000000001 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870316+0000", 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5143999999999997e-05 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870397+0000", 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1118000000000005e-05 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69733 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7290880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:27:01.724 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 59.829108126999998, 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049266982000000001, 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69733 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821302+0000", 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821304+0000", 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821311+0000", 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729999999999998e-06 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821317+0000", 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2660000000000001e-06 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870426+0000", 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049109685 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870497+0000", 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0860999999999998e-05 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870568+0000", 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0984999999999999e-05 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:01.728 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:01.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:27:01.730 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:01.737+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:01.830 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:01.838+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:01.840 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:01.840 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:01.840 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:01.840 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:01.849 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:27:01.933 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:01.940+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:01.998 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:01.998 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:01.999 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:01.999 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:02.007 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:27:02.032 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:02.038+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:02.132 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:02.139+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 110.447123121, 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.166 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.168 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:27:02.170 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 110.177429584, 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:27:02.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 97.033190121999993, 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.9095437, 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.180 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:27:02.181 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.182 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.183 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.184 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.909126822999994, 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:27:02.185 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.186 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.695005426999998, 2024-05-08T08:27:02.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:27:02.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 92.402791597000004, 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.193 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:27:02.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:27:02.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.675589560999995, 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:27:02.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:27:02.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 91.456609196000002, 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.202 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:02.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:27:02.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.206 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.968383830999997, 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:27:02.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 79.048119675999999, 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.214 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.900113542, 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:27:02.215 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:27:02.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.809252647999998, 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:27:02.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.428122405000003, 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113547 3.0 e99/57 3:012c0039:::rbd_data.11918629634e.0000000000000279:head v 99'328, mlcod=99'325)", 2024-05-08T08:27:02.227 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.056942238000001, 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011179999999996, 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:02.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115687+0000", 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.293e-06 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115699+0000", 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-05 2024-05-08T08:27:02.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115705+0000", 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3860000000000004e-06 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115753+0000", 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7938000000000001e-05 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115770+0000", 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6809000000000001e-05 2024-05-08T08:27:02.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119940+0000", 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041703799999999996 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119982+0000", 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113549 3.9 e99/18 3:90ba2507:::rbd_data.11918629634e.000000000000027b:head v 99'310, mlcod=99'308)", 2024-05-08T08:27:02.231 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.054642657999999, 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047200790000000003, 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117981+0000", 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117988+0000", 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0709999999999997e-06 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118003+0000", 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4701999999999999e-05 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118010+0000", 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5619999999999998e-06 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118059+0000", 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8198999999999997e-05 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118074+0000", 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5262000000000001e-05 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122655+0000", 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045809400000000004 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122700+0000", 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:114608 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-05-08T08:27:02.235 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 19.057884617999999, 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95824956900000002, 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114608 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114740+0000", 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5069999999999999e-06 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114747+0000", 2024-05-08T08:27:02.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114762+0000", 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5464999999999998e-05 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114770+0000", 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8599999999999993e-06 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114842+0000", 2024-05-08T08:27:02.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1748999999999994e-05 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114856+0000", 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3976e-05 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071293+0000", 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95643727300000003 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071421+0000", 2024-05-08T08:27:02.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127718 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071573+0000", 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015250100000000001 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072356+0000", 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000783141 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072850+0000", 2024-05-08T08:27:02.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049327699999999997 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072933+0000", 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2993999999999999e-05 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072988+0000", 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5170000000000002e-05 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:543 3.0 e109/57 3:04449ea6:::rbd_data.1191f60358f6.000000000000000c:14 v 109'338, mlcod=102'335)", 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.076310524, 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037001569999999999, 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096313+0000", 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096319+0000", 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.251e-06 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096332+0000", 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3107e-05 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096341+0000", 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096414+0000", 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3201000000000006e-05 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096437+0000", 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2891000000000002e-05 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.099956+0000", 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035193490000000002 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.100013+0000", 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6258000000000002e-05 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:544 3.0 e109/57 3:02813c57:::rbd_data.1191f60358f6.0000000000000058:14 v 109'340, mlcod=102'335)", 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 17.075540012000001, 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057648059999999999, 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097084+0000", 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097092+0000", 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5189999999999992e-06 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097105+0000", 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2685e-05 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097112+0000", 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0049999999999998e-06 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097176+0000", 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4166999999999997e-05 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097203+0000", 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000001e-05 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102791+0000", 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055883590000000002 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102848+0000", 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6789999999999997e-05 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:115393 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 8.1354243690000008, 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036588620000000001, 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115393 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.250 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037200+0000", 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.86e-06 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037209+0000", 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7090000000000008e-06 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037229+0000", 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0049000000000001e-05 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037238+0000", 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6640000000000004e-06 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037305+0000", 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7300999999999998e-05 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037311+0000", 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1029999999999998e-06 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039469+0000", 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021582839999999999 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039564+0000", 2024-05-08T08:27:02.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4547000000000001e-05 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039753+0000", 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018868800000000001 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040757+0000", 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010045869999999999 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:02.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040784+0000", 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6840999999999999e-05 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040824+0000", 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9956000000000002e-05 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040857+0000", 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3272999999999998e-05 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.255 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:02.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:27:02.257 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:02.263+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:02.357 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:02.364+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:02.407 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:02.407 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:02.408 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:02.416 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:27:02.458 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:02.465+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:02.526 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:02.526 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:02.527 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:02.527 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:02.535 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:27:02.578 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:02.586+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493617649000001, 2024-05-08T08:27:02.647 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.648 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.649 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:27:02.650 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493607564000001, 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:27:02.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:02.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493597085000005, 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493580159000004, 2024-05-08T08:27:02.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:27:02.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:27:02.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493559239000007, 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:02.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493520923999995, 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:02.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493497177999998, 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.672 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:27:02.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:27:02.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493474335000002, 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.676 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.677 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493449945999998, 2024-05-08T08:27:02.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.681 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:02.684 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493428026000004, 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:27:02.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:27:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.691 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493406792000002, 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:02.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:27:02.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:27:02.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.695 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.493385348000004, 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:27:02.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:27:02.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.492401044000005, 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.703 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.492379842999995, 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.704 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:27:02.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:27:02.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.707 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.492368373000005, 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.708 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:27:02.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:27:02.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:27:02.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.492358612999993, 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:27:02.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.492348824000004, 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:27:02.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:27:02.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.4923383, 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 70.492328619999995, 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.269067471, 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:02.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:27:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:27:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:02.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:02.733 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:02.734 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:27:02.735 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:02.742+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:02.836 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:02.843+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:02.845 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:02.845 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:02.845 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:27:02.845 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:02.853 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:27:02.937 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:02.944+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:02.959 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:27:02.960 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:27:02.960 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:02.969 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:27:03.037 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:03.044+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:03.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T08:27:03.060 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7] out_osds: [6] dead_osds: [4] live_osds: [0, 2, 3, 5, 6, 7, 1] 2024-05-08T08:27:03.060 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:27:03.060 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-05-08T08:27:03.060 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 6 2024-05-08T08:27:03.080 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:03.080 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:27:03.080 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:27:03.080 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 93.426149272999993, 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.081 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.082 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.083 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:27:03.084 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.085 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.172668251999994, 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.086 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:27:03.087 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.088 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.089 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.172616304000002, 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.090 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:27:03.091 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:27:03.092 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.093 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.172581449000006, 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.094 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.095 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.096 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.097 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.172551185000003, 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.098 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:27:03.099 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:27:03.100 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.101 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.171241819000002, 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.102 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:27:03.103 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:27:03.104 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.105 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.171221056999997, 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:03.106 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:27:03.107 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:27:03.108 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.109 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.171149587000002, 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.110 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:27:03.111 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:27:03.112 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.113 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.171122218999997, 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.114 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.115 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:27:03.116 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:27:03.117 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.169167829000003, 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.118 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.119 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.120 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.121 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.169148261999993, 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.122 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.123 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.124 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.125 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:03.126 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.169128491999999, 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.127 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.128 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.129 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:27:03.130 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.169109493999997, 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.131 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.132 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:27:03.133 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.167209858000007, 2024-05-08T08:27:03.134 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.135 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:27:03.136 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:27:03.137 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.138 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.167189891000007, 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.139 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.140 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:27:03.141 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.142 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.167168317999995, 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.143 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:27:03.144 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.145 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:27:03.146 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.167044328000003, 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.147 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.148 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.149 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.150 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.167015532999997, 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.151 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.152 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.153 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.154 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.165097996, 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.155 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.156 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.157 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.158 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 71.165076541999994, 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.159 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:27:03.160 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.161 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:27:03.162 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.163 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.163 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.163 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.163 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.163 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:03.164 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:27:03.165 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:03.171+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:03.265 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:03.273+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:03.312 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:03.312 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:03.312 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:27:03.312 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:03.321 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:27:03.366 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:03.374+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:03.427 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:03.427 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:03.427 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:27:03.427 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:27:03.427 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:03.436 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:27:03.480 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:03.487+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:03.493 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:27:03.499 INFO:tasks.ceph.mon.a.smithi031.stderr:2024-05-08T08:27:03.507+0000 7f2e11ed7640 -1 mon.a@0(leader).osd e127 definitely_dead 0 2024-05-08T08:27:03.543 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:03.543 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770272857999998, 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:27:03.544 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:27:03.545 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:27:03.546 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:27:03.547 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770252139999997, 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.548 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.549 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.550 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.551 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770224202999998, 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.552 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:27:03.553 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:27:03.554 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.555 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770202439999998, 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.556 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.557 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.558 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770179861999999, 2024-05-08T08:27:03.559 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.560 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.561 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.562 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770139792999998, 2024-05-08T08:27:03.563 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.564 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:27:03.565 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:27:03.566 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770117485, 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:27:03.567 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:27:03.568 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:27:03.569 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.570 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770088903999998, 2024-05-08T08:27:03.571 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.572 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:27:03.573 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:27:03.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770064650999998, 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.575 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:27:03.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.770032252999997, 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.579 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.768694447999998, 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.583 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.584 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.768672430000002, 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.587 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:27:03.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.768630070999997, 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.591 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.592 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:27:03.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.768588551999997, 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:27:03.595 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:27:03.596 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:27:03.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.766610456000002, 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.599 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.600 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:27:03.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.766599161999999, 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:27:03.603 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.604 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:27:03.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:27:03.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.762511385000003, 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.607 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.608 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:27:03.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:27:03.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.762495907999998, 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.611 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.757377132999999, 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.615 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:27:03.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:27:03.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 61.757099203999999, 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:03.619 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:03.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:03.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:27:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:27:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:03.623 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:03.623 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:03.624 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:27:03.624 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:27:03.624 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:03.631+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:03.625 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:27:03.638 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:27:03.638 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:27:03.655 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:27:03.655 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:27:03.672 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:27:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:27:03.672 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:27:03.690 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:27:03.690 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:27:03.708 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:27:03.709 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:27:03.725 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:03.731+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:03.726 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:27:03.726 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:27:03.729 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:27:03.730 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:27:03.772 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:03.772 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:03.781 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:27:03.865 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:03.872+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:03.926 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:03.933+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:03.927 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:03.927 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:03.936 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:27:04.027 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:04.033+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:04.043 INFO:teuthology.orchestra.run.smithi031.stderr:marked in osd.6. 2024-05-08T08:27:04.046 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is set 2024-05-08T08:27:04.065 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-05-08T08:27:04.087 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:04.087 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:04.096 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:27:04.150 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:04.158+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:04.219 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:04.219 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:04.228 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:27:04.229 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:04.237+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:04.330 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:04.338+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:04.378 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:04.378 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:04.387 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:27:04.431 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:04.439+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:04.532 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:04.539+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:04.536 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:04.536 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:04.545 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:27:04.633 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:04.640+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:04.695 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:04.695 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:04.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:27:04.747 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:04.755+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:04.814 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:04.814 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:04.823 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:27:04.863 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:04.871+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:04.935 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:04.943+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:04.972 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:04.972 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:04.981 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:27:05.065 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:05.073+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:05.137 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:05.144+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:05.137 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:05.137 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:05.146 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:27:05.238 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:05.245+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:05.295 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:05.295 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:05.304 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:27:05.346 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:05.353+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:05.439 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:05.447+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:05.454 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:05.463 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:27:05.547 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:05.555+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:05.613 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:05.614 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:05.622 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:27:05.664 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:05.671+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:05.730 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:27:05.772 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:05.772 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:05.773 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:05.780+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:05.782 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:27:05.844 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:05.851+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:05.857 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:27:05.858 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:27:05.875 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:27:05.875 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:27:05.893 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:27:05.893 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:27:05.910 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:27:05.911 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:27:05.911 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:27:05.928 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:27:05.929 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781490172000005, 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.935 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:05.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:27:05.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781480087000006, 2024-05-08T08:27:05.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:05.943 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781469607999995, 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:27:05.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.947 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781452681999994, 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:27:05.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:05.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:27:05.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:27:05.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781431761999997, 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.952 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:05.956 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781393446999999, 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.958 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:27:05.960 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781369701000003, 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.961 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.962 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781346858000006, 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.965 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:27:05.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:27:05.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:05.969 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781322469000003, 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.973 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781300548999994, 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.974 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:27:05.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.977 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781279315000006, 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.978 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.979 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:27:05.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.781257870999994, 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.982 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.983 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.984 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.985 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.780273566999995, 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:05.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:27:05.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:27:05.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.780252365999999, 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:05.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.780240895999995, 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:27:05.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:27:05.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:27:05.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.780231135999998, 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:05.999 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:27:06.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.003 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.780221346999994, 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:06.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:27:06.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.780210823000004, 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:06.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:27:06.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:27:06.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 73.780201142999999, 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:06.012 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.013 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:27:06.014 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.015 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:06.016 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 50.556939993999997, 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.017 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:27:06.018 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:27:06.019 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:06.020 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:27:06.021 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:27:06.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:06.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.021 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:06.021 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:06.021 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:06.023 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:27:06.023 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:27:06.023 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:27:06.024 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:27:06.024 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:27:06.024 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:27:06.025 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:27:06.025 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:06.030+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:06.067 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T08:27:06.123 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:06.131+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:06.164 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:06.164 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:27:06.164 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.164 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:116747 3.17 3:e9728b91:::rbd_header.11917ede465c:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e130)", 2024-05-08T08:27:06.164 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:27:06.170182+0000", 2024-05-08T08:27:06.164 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 0.002035799, 2024-05-08T08:27:06.164 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0020591860000000002, 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 116747 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170182+0000", 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.165 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170182+0000", 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170183+0000", 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8999999999999995e-07 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.166 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170187+0000", 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.693e-06 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170197+0000", 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5980000000000003e-06 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170203+0000", 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8329999999999996e-06 2024-05-08T08:27:06.167 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170239+0000", 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5738999999999998e-05 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:27:06.170281+0000", 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1770999999999997e-05 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: ], 2024-05-08T08:27:06.168 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 1 2024-05-08T08:27:06.169 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:06.175 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:27:06.224 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:06.231+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:06.295 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:06.295 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:27:06.296 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:27:06.296 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:06.305 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:27:06.346 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:06.353+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:06.426 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:06.433+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:06.454 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:06.454 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:27:06.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:27:06.454 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:21944 3.c 3:30cf8b0f:::rbd_data.1191daf6073a.0000000000000000:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 33554432 write_size 33554432,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 96.799994225999995, 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.054423933000000001, 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 21944 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660828+0000", 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.660829+0000", 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6339999999999999e-06 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.456 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674565+0000", 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.013735799 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674591+0000", 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5316999999999998e-05 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674618+0000", 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7389000000000002e-05 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674698+0000", 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0416999999999999e-05 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.674820+0000", 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000121508 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 3", 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.685546+0000", 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.010726484 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715066+0000", 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.029519255000000001 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715120+0000", 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4534999999999999e-05 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715191+0000", 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0568000000000006e-05 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:29.715252+0000", 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1026999999999997e-05 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49387 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 95232~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.546513204999997, 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052994853000000001, 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49387 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914309+0000", 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3099999999999998e-07 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914323+0000", 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3397000000000001e-05 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914334+0000", 2024-05-08T08:27:06.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1063000000000001e-05 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914340+0000", 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8889999999999999e-06 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967116+0000", 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052776357000000003 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967200+0000", 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.3836000000000002e-05 2024-05-08T08:27:06.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967304+0000", 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010388 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49388 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 104448~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.546461257000004, 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053148931000000003, 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.464 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49388 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914361+0000", 2024-05-08T08:27:06.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914362+0000", 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4000000000000001e-07 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914366+0000", 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9759999999999997e-06 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914377+0000", 2024-05-08T08:27:06.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1038e-05 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914384+0000", 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6559999999999992e-06 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967343+0000", 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052959147999999998 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967418+0000", 2024-05-08T08:27:06.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4895999999999996e-05 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967510+0000", 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1477000000000005e-05 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49389 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 113664~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.546426401999994, 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053364976000000001, 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.468 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49389 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914396+0000", 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914399+0000", 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.565e-06 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914412+0000", 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2928000000000001e-05 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914418+0000", 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0700000000000003e-06 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967551+0000", 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053133706000000003 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967644+0000", 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2898000000000004e-05 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967761+0000", 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000116328 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49390 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 122880~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.546396138000006, 2024-05-08T08:27:06.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053474714999999999, 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49390 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.914426+0000", 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915630+0000", 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0012038940000000001 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915634+0000", 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3549999999999998e-06 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915645+0000", 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0993e-05 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915654+0000", 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4950000000000008e-06 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967793+0000", 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052139061 2024-05-08T08:27:06.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967838+0000", 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5529999999999999e-05 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.967901+0000", 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2386999999999997e-05 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.476 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49394 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 161792~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.545086772000005, 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972052999999998, 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49394 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915735+0000", 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915736+0000", 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8700000000000001e-07 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915738+0000", 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915744+0000", 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.229e-06 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915749+0000", 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1749999999999996e-06 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968491+0000", 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052741982999999999 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968582+0000", 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.1107999999999996e-05 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968707+0000", 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00012496499999999999 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.480 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49395 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 171008~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.545066009999999, 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053157488000000003, 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49395 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.481 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915756+0000", 2024-05-08T08:27:06.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6100000000000002e-07 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915759+0000", 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2120000000000002e-06 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915770+0000", 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0916e-05 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915775+0000", 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6829999999999996e-06 2024-05-08T08:27:06.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968748+0000", 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052972798000000001 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968823+0000", 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5035000000000002e-05 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968914+0000", 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0482999999999994e-05 2024-05-08T08:27:06.484 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49396 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 180224~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.544994540000005, 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053349098999999997, 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.485 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49396 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.486 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915828+0000", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2200000000000004e-07 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915831+0000", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.706e-06 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915839+0000", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4980000000000006e-06 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915846+0000", 2024-05-08T08:27:06.487 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5220000000000004e-06 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.968989+0000", 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053143207999999997 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969074+0000", 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5461000000000001e-05 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969177+0000", 2024-05-08T08:27:06.488 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010218200000000001 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49397 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 189440~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.544967172, 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053535992999999997, 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.489 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49397 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.915855+0000", 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.490 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917631+0000", 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017761090000000001 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917636+0000", 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7520000000000004e-06 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917647+0000", 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1318000000000001e-05 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917655+0000", 2024-05-08T08:27:06.491 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9000000000000006e-06 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969218+0000", 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051563367999999998 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969296+0000", 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8157000000000004e-05 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.969391+0000", 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.4388999999999994e-05 2024-05-08T08:27:06.492 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49404 3.5 3:a2bd85ef:::rbd_data.11914ad0077c.0000000000000000:1a [read 253952~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.543012782000005, 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053085319999999998, 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49404 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.493 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917809+0000", 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.494 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917810+0000", 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3599999999999999e-07 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917812+0000", 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.182e-06 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917818+0000", 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.495 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917822+0000", 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6670000000000004e-06 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970695+0000", 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052872937000000002 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970784+0000", 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9056999999999998e-05 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970895+0000", 2024-05-08T08:27:06.496 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000110232 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49405 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 1024~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.542993214999996, 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053257839000000001, 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.497 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49405 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.498 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917829+0000", 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9700000000000002e-07 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917831+0000", 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2000000000000001e-06 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917837+0000", 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9290000000000003e-06 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.499 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917842+0000", 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8339999999999998e-06 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970920+0000", 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053077892000000002 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.970996+0000", 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.6205000000000006e-05 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.500 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971087+0000", 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0382000000000003e-05 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49406 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 10240~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.542973445000001, 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053488638999999998, 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.501 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49406 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.502 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917849+0000", 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1199999999999998e-07 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917851+0000", 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.294e-06 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917857+0000", 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8420000000000001e-06 2024-05-08T08:27:06.503 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917862+0000", 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7210000000000002e-06 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971130+0000", 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053268497999999997 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971222+0000", 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2108999999999995e-05 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.504 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971337+0000", 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000114763 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49407 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 19456~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.542954447, 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053701741999999997, 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.505 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49407 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.917868+0000", 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.506 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919632+0000", 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017641899999999999 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919636+0000", 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.532e-06 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919648+0000", 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1409999999999999e-05 2024-05-08T08:27:06.507 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919656+0000", 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.7829999999999999e-06 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971381+0000", 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051725225 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971467+0000", 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6533000000000007e-05 2024-05-08T08:27:06.508 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.971569+0000", 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000102069 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49412 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 65536~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.541054810999995, 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052895351, 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.509 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49412 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.510 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919767+0000", 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919768+0000", 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9299999999999999e-07 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919770+0000", 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9360000000000002e-06 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919776+0000", 2024-05-08T08:27:06.511 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1920000000000002e-06 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919781+0000", 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9729999999999997e-06 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972475+0000", 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052694522000000001 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.512 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972553+0000", 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8238999999999994e-05 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972663+0000", 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000109096 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49413 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 74752~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:27:06.513 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.541034843999995, 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053073756999999999, 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49413 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.514 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919787+0000", 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919788+0000", 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8500000000000002e-07 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.515 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919790+0000", 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9750000000000001e-06 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919797+0000", 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5399999999999998e-06 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919802+0000", 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2800000000000003e-06 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.516 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972693+0000", 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052890606 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972769+0000", 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5899e-05 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972861+0000", 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2071999999999995e-05 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.517 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49414 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 83968~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.541013270999997, 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053245959000000002, 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49414 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.518 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919809+0000", 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.46e-07 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.519 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919811+0000", 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.057e-06 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919817+0000", 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2369999999999997e-06 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919823+0000", 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1830000000000002e-06 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.520 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972891+0000", 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053068825999999999 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.972965+0000", 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3298000000000002e-05 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973055+0000", 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0012000000000005e-05 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.521 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49415 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 93184~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.540889281000005, 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053310614999999999, 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49415 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.522 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919933+0000", 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5400000000000001e-07 2024-05-08T08:27:06.523 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919936+0000", 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.1709999999999998e-06 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919946+0000", 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0559999999999994e-06 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919952+0000", 2024-05-08T08:27:06.524 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8229999999999997e-06 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973084+0000", 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053131992000000003 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973157+0000", 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2890000000000002e-05 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973243+0000", 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6129000000000002e-05 2024-05-08T08:27:06.525 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49416 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 102400~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.540860486, 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053499573000000002, 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.526 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49416 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.919962+0000", 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.527 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921665+0000", 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00170319 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921669+0000", 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.172e-06 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921680+0000", 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1299e-05 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.528 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921688+0000", 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9500000000000001e-06 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973274+0000", 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051585606999999999 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973371+0000", 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.7371e-05 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.973461+0000", 2024-05-08T08:27:06.529 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.9983999999999998e-05 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49424 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 168960~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.538942949000003, 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052949081000000002, 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.530 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49424 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921879+0000", 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.531 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921880+0000", 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6699999999999999e-07 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921882+0000", 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0530000000000001e-06 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921888+0000", 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7089999999999996e-06 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.532 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921893+0000", 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0640000000000001e-06 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974654+0000", 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.052761095000000001 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974730+0000", 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.5185999999999995e-05 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974828+0000", 2024-05-08T08:27:06.533 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8506999999999997e-05 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:49425 3.5 3:a18910ab:::rbd_data.11914ad0077c.0000000000000001:1a [read 178176~1024] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 74.538921494999997, 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.053067005, 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.534 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 49425 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.921901+0000", 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.535 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923631+0000", 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001730629 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923636+0000", 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5249999999999999e-06 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923648+0000", 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2079000000000001e-05 2024-05-08T08:27:06.536 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.923655+0000", 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.588e-06 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974855+0000", 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.051199230999999998 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974906+0000", 2024-05-08T08:27:06.537 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1740000000000003e-05 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:25:51.974968+0000", 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1212999999999998e-05 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.538 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:06.540 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:27:06.540 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:06.547+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:06.641 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:06.647+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:06.647 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:06.647 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:06.647 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:27:06.647 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:06.656 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:27:06.742 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:06.750+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:06.802 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:06.802 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:06.802 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:27:06.802 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:27:06.802 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:06.810 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:27:06.843 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:06.851+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69223 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1359872~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144384592999998, 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043564888000000003, 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.918 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69223 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.919 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780206+0000", 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780207+0000", 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9900000000000001e-07 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780209+0000", 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3709999999999998e-06 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.920 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780215+0000", 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.409e-06 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780220+0000", 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6090000000000003e-06 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823649+0000", 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043428864999999997 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.921 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823690+0000", 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0905999999999998e-05 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823771+0000", 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1229000000000004e-05 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.922 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69224 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1589248~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144363874999996, 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043661459, 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69224 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.923 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780227+0000", 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.924 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780229+0000", 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.9769999999999999e-06 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780239+0000", 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5719999999999992e-06 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780244+0000", 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1900000000000003e-06 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.925 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823798+0000", 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043554392999999997 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823836+0000", 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7296999999999999e-05 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823888+0000", 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2629e-05 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.926 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69225 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 1818624~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144335937999998, 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043778872000000003, 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69225 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.927 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780255+0000", 2024-05-08T08:27:06.928 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.3500000000000002e-07 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780258+0000", 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.464e-06 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780264+0000", 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.4239999999999998e-06 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780269+0000", 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7899999999999999e-06 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.929 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823919+0000", 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0436497 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.823959+0000", 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0438999999999997e-05 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824034+0000", 2024-05-08T08:27:06.930 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.462e-05 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69226 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144314175000005, 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043872547999999997, 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.931 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69226 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.932 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780277+0000", 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4099999999999999e-07 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780279+0000", 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2180000000000002e-06 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780287+0000", 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.0399999999999993e-06 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.933 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780292+0000", 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.028e-06 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824061+0000", 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043768286000000003 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824097+0000", 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6332999999999999e-05 2024-05-08T08:27:06.934 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824149+0000", 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2201999999999999e-05 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69227 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2277376~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144291597000006, 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043990833999999999, 2024-05-08T08:27:06.935 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69227 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.936 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780299+0000", 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780300+0000", 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1399999999999997e-07 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780302+0000", 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3300000000000001e-06 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.937 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780311+0000", 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7390000000000005e-06 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780316+0000", 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0100000000000003e-06 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824180+0000", 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043864411999999998 2024-05-08T08:27:06.938 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824220+0000", 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9854000000000002e-05 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824290+0000", 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0074999999999996e-05 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.939 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69228 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2506752~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144251527999998, 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044067304000000002, 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69228 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.940 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780339+0000", 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780340+0000", 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0100000000000005e-07 2024-05-08T08:27:06.941 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780342+0000", 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4439999999999998e-06 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780349+0000", 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7689999999999999e-06 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.942 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780354+0000", 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.916e-06 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824320+0000", 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043965873000000003 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824356+0000", 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6319000000000002e-05 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824406+0000", 2024-05-08T08:27:06.943 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0382e-05 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69229 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2736128~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.14422922, 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044185974000000003, 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.944 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69229 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780361+0000", 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.945 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780362+0000", 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5000000000000003e-07 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780364+0000", 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0789999999999999e-06 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.946 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780370+0000", 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0129999999999999e-06 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780382+0000", 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2170999999999999e-05 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824435+0000", 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044053059999999998 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.947 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824476+0000", 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0219999999999998e-05 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824547+0000", 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.1880999999999998e-05 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69230 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 2965504~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144200639000005, 2024-05-08T08:27:06.948 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044290826999999998, 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69230 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.949 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780390+0000", 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780391+0000", 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.34e-07 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780393+0000", 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.3609999999999999e-06 2024-05-08T08:27:06.950 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780402+0000", 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2439999999999999e-06 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780407+0000", 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2039999999999996e-06 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824576+0000", 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044168517999999997 2024-05-08T08:27:06.951 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824613+0000", 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6798999999999998e-05 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824681+0000", 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8267000000000002e-05 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.952 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69231 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144176385999998, 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044402380999999998, 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69231 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.953 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780414+0000", 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780415+0000", 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.51e-07 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.954 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780423+0000", 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2139999999999996e-06 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780430+0000", 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.2389999999999996e-06 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780436+0000", 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.1319999999999999e-06 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.955 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824711+0000", 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044274221000000002 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824749+0000", 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8497000000000001e-05 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824817+0000", 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7626999999999995e-05 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.956 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69232 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 3424256~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.144143987999996, 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044486045000000002, 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69232 2024-05-08T08:27:06.957 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780447+0000", 2024-05-08T08:27:06.958 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.4900000000000001e-07 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780449+0000", 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.311e-06 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780456+0000", 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8920000000000003e-06 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.780465+0000", 2024-05-08T08:27:06.959 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.157e-06 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824846+0000", 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044381177000000001 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824882+0000", 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6109999999999998e-05 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.824933+0000", 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0949000000000003e-05 2024-05-08T08:27:06.960 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69236 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4341760~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.142806183000005, 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043646695999999999, 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.961 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69236 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.962 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781785+0000", 2024-05-08T08:27:06.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9100000000000004e-07 2024-05-08T08:27:06.963 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.963 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.963 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.963 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781788+0000", 2024-05-08T08:27:06.963 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5859999999999999e-06 2024-05-08T08:27:06.965 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.965 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.965 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.965 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781795+0000", 2024-05-08T08:27:06.965 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0670000000000003e-06 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781800+0000", 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0849999999999996e-06 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825345+0000", 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043545493999999997 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.966 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825381+0000", 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5654000000000001e-05 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825431+0000", 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0318999999999997e-05 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69237 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4571136~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.142784164999995, 2024-05-08T08:27:06.967 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043749488000000003, 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69237 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.968 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781807+0000", 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2e-07 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781809+0000", 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.934e-06 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.969 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781815+0000", 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6250000000000004e-06 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781819+0000", 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825461+0000", 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043641709000000001 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.970 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825500+0000", 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8889e-05 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825556+0000", 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6107000000000003e-05 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69238 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4800512~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:27:06.971 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.142741806000004, 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04392778, 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69238 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.972 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781849+0000", 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781850+0000", 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2600000000000002e-07 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781852+0000", 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4849999999999999e-06 2024-05-08T08:27:06.973 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781859+0000", 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9820000000000002e-06 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781875+0000", 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5798e-05 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825594+0000", 2024-05-08T08:27:06.974 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043718869 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825675+0000", 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1071999999999999e-05 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825777+0000", 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00010194799999999999 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.975 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69239 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5029888~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.142700286999997, 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.044027148000000002, 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69239 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:27:06.976 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.781890+0000", 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783630+0000", 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0017398540000000001 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783634+0000", 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.817e-06 2024-05-08T08:27:06.977 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783648+0000", 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3438000000000001e-05 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783654+0000", 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8940000000000004e-06 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825816+0000", 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.042161654 2024-05-08T08:27:06.978 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825858+0000", 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2279e-05 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.825918+0000", 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9212000000000002e-05 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.979 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69248 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7094272~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.140722190999995, 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043647123000000003, 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69248 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.980 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783869+0000", 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.7399999999999999e-07 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.981 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783870+0000", 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3149999999999999e-06 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783873+0000", 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2909999999999999e-06 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783876+0000", 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.6419999999999999e-06 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.982 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827407+0000", 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043531090000000001 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827462+0000", 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5072999999999999e-05 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827516+0000", 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3437999999999999e-05 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.983 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69249 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 7323648~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.140710897000005, 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043852965000000001, 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69249 2024-05-08T08:27:06.984 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.783880+0000", 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785632+0000", 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001751797 2024-05-08T08:27:06.985 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785636+0000", 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.622e-06 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785646+0000", 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.3260000000000002e-06 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.785650+0000", 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9069999999999998e-06 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.986 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827563+0000", 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041912061 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827653+0000", 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.0789000000000001e-05 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.827733+0000", 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.9462999999999995e-05 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.987 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69371 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5308416~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.136623119999996, 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043630805000000002, 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69371 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.988 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787968+0000", 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.8999999999999998e-07 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.989 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787969+0000", 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5659999999999999e-06 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787974+0000", 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9200000000000003e-06 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787978+0000", 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.997e-06 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.990 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831517+0000", 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043538614000000003 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831557+0000", 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0229999999999999e-05 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831598+0000", 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1187999999999999e-05 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.991 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69372 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 5537792~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.136607643000005, 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043806907999999999, 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69372 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.992 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.787983+0000", 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789632+0000", 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001649237 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.993 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789637+0000", 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6709999999999998e-06 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789654+0000", 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.6949e-05 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789666+0000", 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2480000000000001e-05 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831637+0000", 2024-05-08T08:27:06.994 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041970911 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831712+0000", 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4382999999999996e-05 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831790+0000", 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.8276999999999996e-05 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.995 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69494 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4177920~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.131488868000005, 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043631072999999999, 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69494 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:27:06.996 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793102+0000", 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793338+0000", 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000236562 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793341+0000", 2024-05-08T08:27:06.997 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0699999999999998e-06 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793351+0000", 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.5179999999999993e-06 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793355+0000", 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2599999999999999e-06 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:06.998 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836578+0000", 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043222674000000003 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836664+0000", 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.5625999999999996e-05 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836733+0000", 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.9363000000000007e-05 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:06.999 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69496 3.10 3:0870dc17:::rbd_data.1191d80f8e36.0000000000000001:1e [read 4636672~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.131210938999999, 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043604679, 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69496 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.000 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793380+0000", 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793542+0000", 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000162175 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.001 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793546+0000", 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0409999999999999e-06 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793555+0000", 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.2269999999999999e-06 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.793561+0000", 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.7899999999999996e-06 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.002 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836891+0000", 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.043329920000000001 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836934+0000", 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.2917000000000003e-05 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.836984+0000", 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0609000000000003e-05 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.003 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.004 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:07.005 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:27:07.005 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:07.012+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:07.070 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is set 2024-05-08T08:27:07.122 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:07.122 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:07.122 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:27:07.122 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:07.132 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:27:07.133 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:07.141+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:07.206 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:07.213+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:07.240 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:07.240 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "complaint_time": 30, 2024-05-08T08:27:07.241 INFO:teuthology.orchestra.run.smithi149.stdout: "num_blocked_ops": 0 2024-05-08T08:27:07.241 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:07.250 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:27:07.334 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:07.341+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:27:07.399 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 129.84015602700001, 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.400 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.401 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:27:07.402 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:27:07.403 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.404 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.616050720000004, 2024-05-08T08:27:07.405 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.406 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.407 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.408 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615965566, 2024-05-08T08:27:07.409 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.410 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:27:07.411 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.412 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:07.413 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.61593766, 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.414 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:27:07.415 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.416 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.417 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615909259000006, 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.418 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.419 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.420 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615825119999997, 2024-05-08T08:27:07.421 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.422 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.423 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:27:07.424 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.425 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615744625000005, 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:27:07.426 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.427 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:27:07.428 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:27:07.429 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615712044999995, 2024-05-08T08:27:07.430 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.431 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:27:07.432 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:27:07.433 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.434 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615658854000003, 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.435 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.436 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.437 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:27:07.438 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615632875000003, 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.439 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.440 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:27:07.441 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.442 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:07.443 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615577543000001, 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.444 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:27:07.445 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.446 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.447 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615427510000004, 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.448 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:27:07.449 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.450 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.451 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615377851000005, 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.452 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.453 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.454 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.455 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615343281999998, 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.456 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.457 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.458 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:27:07.459 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615321274999999, 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.460 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.461 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.462 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:27:07.463 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615298300999996, 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:27:07.464 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.465 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.466 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.467 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:27:07.468 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.615268623000006, 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.469 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.470 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.471 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.472 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.614046044999995, 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.473 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.474 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:27:07.475 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.476 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.614035551000001, 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:07.477 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.478 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:27:07.479 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:07.480 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 65.608916715000007, 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:07.481 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:07.482 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:27:07.483 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:07.484 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:07.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:27:07.486 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:07.492+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:07.586 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:07.594+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:07.603 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:07.603 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:07.603 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:07.603 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:07.612 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:27:07.696 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:07.704+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:07.762 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:07.763 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:07.763 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:07.771 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:27:07.788 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:07.795+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:07.889 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:07.896+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:07.924 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:07.924 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.770604813000006, 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:07.925 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.926 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.927 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.928 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.770583712999994, 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.929 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.930 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.931 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.932 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 75.770563054999997, 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:27:07.933 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:07.934 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:27:07.935 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:27:07.936 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108422 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936559178000003, 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032000842000000002, 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.937 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108422 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.938 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994307+0000", 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994314+0000", 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-05-08T08:27:07.939 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994320+0000", 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7760000000000003e-06 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026109+0000", 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031789339 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026194+0000", 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4841999999999998e-05 2024-05-08T08:27:07.940 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026301+0000", 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106667 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108428 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936531569000003, 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032197602999999998, 2024-05-08T08:27:07.941 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108428 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.942 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994328+0000", 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994330+0000", 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5689999999999999e-06 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.943 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994338+0000", 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5830000000000001e-06 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994343+0000", 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026333+0000", 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031990022 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.944 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026419+0000", 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6278000000000003e-05 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026525+0000", 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010569 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108429 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936508289000002, 2024-05-08T08:27:07.945 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032394017999999997, 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108429 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:07.946 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994353+0000", 2024-05-08T08:27:07.947 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3829999999999999e-06 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994360+0000", 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6850000000000004e-06 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994365+0000", 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2680000000000002e-06 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026559+0000", 2024-05-08T08:27:07.948 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032193927999999997 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026647+0000", 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7882999999999999e-05 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026745+0000", 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7460999999999994e-05 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.949 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108430 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3211264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936486320999997, 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032603935000000001, 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108430 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.950 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.951 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994375+0000", 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3989999999999998e-06 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994383+0000", 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2869999999999999e-06 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994387+0000", 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.952 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026777+0000", 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032389793 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026865+0000", 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7763999999999995e-05 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026976+0000", 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111488 2024-05-08T08:27:07.953 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108431 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3244032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936463914000001, 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032796189000000003, 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.954 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108431 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.955 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994398+0000", 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994404+0000", 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3969999999999999e-06 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.956 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994410+0000", 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027010+0000", 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032600259999999999 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027094+0000", 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128999999999994e-05 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027191+0000", 2024-05-08T08:27:07.957 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7089000000000005e-05 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108444 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936442044000003, 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032973852999999997, 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.958 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108444 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.959 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994419+0000", 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2900000000000001e-06 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994426+0000", 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5080000000000002e-06 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.960 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994431+0000", 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0200000000000002e-06 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027223+0000", 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032791703999999998 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027303+0000", 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0068999999999995e-05 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.961 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027391+0000", 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7899000000000007e-05 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108445 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936421019999997, 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033150011, 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.962 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108445 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.963 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994441+0000", 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994448+0000", 2024-05-08T08:27:07.964 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9929999999999998e-06 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994452+0000", 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8199999999999996e-06 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027420+0000", 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032967989000000003 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027500+0000", 2024-05-08T08:27:07.965 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9332000000000005e-05 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027588+0000", 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8080999999999999e-05 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108446 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936398761, 2024-05-08T08:27:07.966 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033416003, 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108446 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.967 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994461+0000", 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994463+0000", 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:27:07.968 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994470+0000", 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4489999999999996e-06 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994475+0000", 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027699+0000", 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033223821000000001 2024-05-08T08:27:07.969 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027779+0000", 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0411e-05 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027876+0000", 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6855999999999996e-05 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.970 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108447 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.936376717000002, 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033598876, 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108447 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.971 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994483+0000", 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.972 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994485+0000", 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994492+0000", 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994497+0000", 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3569999999999997e-06 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.973 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027912+0000", 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033415331999999999 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027991+0000", 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8825999999999994e-05 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.028081+0000", 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9858000000000006e-05 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.974 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108675 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.931380797999999, 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032085805000000002, 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108675 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.975 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.976 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999480+0000", 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.793e-06 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999486+0000", 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999490+0000", 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.464e-06 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.977 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031407+0000", 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031916959000000002 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031488+0000", 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0567000000000003e-05 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031564+0000", 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6114999999999999e-05 2024-05-08T08:27:07.978 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108676 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.931363300000001, 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032251646000000002, 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.979 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108676 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.980 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999498+0000", 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999503+0000", 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999509+0000", 2024-05-08T08:27:07.981 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8270000000000003e-06 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031594+0000", 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032084797999999998 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031672+0000", 2024-05-08T08:27:07.982 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8765999999999998e-05 2024-05-08T08:27:07.985 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031747+0000", 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4881000000000004e-05 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108677 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.931344543999998, 2024-05-08T08:27:07.986 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032405850999999999, 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108677 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.987 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999515+0000", 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.79e-07 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999516+0000", 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.988 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999522+0000", 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2460000000000003e-06 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999526+0000", 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.673e-06 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031777+0000", 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032250681000000003 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.989 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031851+0000", 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3694000000000003e-05 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031920+0000", 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9449999999999994e-05 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108678 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.990 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.931327238999998, 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032573421999999998, 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108678 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:07.991 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0800000000000004e-07 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999538+0000", 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-05-08T08:27:07.992 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999545+0000", 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999549+0000", 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.993 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031947+0000", 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032397903999999998 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032029+0000", 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1985999999999997e-05 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032105+0000", 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6032999999999995e-05 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.994 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108703 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6832128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.929999895000002, 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03217747, 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108703 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.995 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:07.996 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.996 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:07.996 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:07.996 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.996 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000861+0000", 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8199999999999999e-06 2024-05-08T08:27:07.997 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000866+0000", 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0880000000000002e-06 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000871+0000", 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510000000000004e-06 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032860+0000", 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031989640999999999 2024-05-08T08:27:07.998 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032962+0000", 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010197200000000001 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033036+0000", 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4299000000000001e-05 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:07.999 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108704 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6864896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.929650791, 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032008086999999998, 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108704 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:08.000 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001209+0000", 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5199999999999996e-07 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:08.001 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001213+0000", 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8770000000000003e-06 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001224+0000", 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1342000000000001e-05 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001231+0000", 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:08.002 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033067+0000", 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031835544 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033149+0000", 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1719999999999997e-05 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033216+0000", 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7645999999999996e-05 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:08.003 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108705 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.929615435000002, 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032150629999999999, 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108705 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.004 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001244+0000", 2024-05-08T08:27:08.005 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001247+0000", 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001254+0000", 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7190000000000001e-06 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:08.006 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001260+0000", 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5099999999999998e-06 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033245+0000", 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031984929000000002 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033322+0000", 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7435999999999995e-05 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:08.007 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033394+0000", 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1740000000000001e-05 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108706 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6930432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 47.929587742000002, 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032385262999999997, 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:08.008 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108706 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.009 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001272+0000", 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001274+0000", 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001281+0000", 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0840000000000003e-06 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.010 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001287+0000", 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033490+0000", 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032203455999999998 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033573+0000", 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2860000000000003e-05 2024-05-08T08:27:08.011 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033656+0000", 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3380999999999993e-05 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:08.012 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:08.014 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:08.021+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:08.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:27:08.026 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:27:08.114 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:08.121+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:08.173 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:08.173 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:08.174 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:08.174 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:08.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:27:08.188 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:27:08.188 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:27:08.207 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:27:08.207 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:27:08.214 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:08.222+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:08.224 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:27:08.225 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:27:08.242 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:27:08.242 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:27:08.242 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:27:08.260 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:27:08.261 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:27:08.279 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:27:08.279 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:27:08.298 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:27:08.298 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:27:08.299 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:08.300 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:08.301 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:27:08.301 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:27:08.309 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:27:08.350 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:08.358+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:08.416 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:08.423+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:08.460 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:08.460 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:08.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:27:08.517 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:08.524+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:08.579 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:08.579 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:08.588 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:27:08.631 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:08.639+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:08.697 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:08.697 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:08.706 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:27:08.748 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:08.755+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:08.820 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:08.826+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:08.857 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:08.857 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:08.865 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:27:08.920 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:08.927+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:08.975 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:08.975 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:08.984 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:27:09.021 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:09.028+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:09.065 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 7, 6] out_osds: [] dead_osds: [4] live_osds: [0, 2, 3, 5, 6, 7, 1] 2024-05-08T08:27:09.065 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T08:27:09.066 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T08:27:09.066 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T08:27:09.093 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:27:09.102 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:09.102 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:09.113 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:27:09.154 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:09.162+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:09.222 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:09.230+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:09.280 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:09.280 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:09.289 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:27:09.331 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:09.339+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:09.401 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:09.401 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "epoch": 133, 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "fsid": "1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "created": "2024-05-08T08:24:44.143941+0000", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "modified": "2024-05-08T08:27:09.082681+0000", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "last_up_change": "2024-05-08T08:26:46.068407+0000", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "last_in_change": "2024-05-08T08:27:04.049694+0000", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_num": 5806080, 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_set": [ 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "nodeep-scrub", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "noscrub", 2024-05-08T08:27:09.404 INFO:teuthology.orchestra.run.smithi031.stdout: "pglog_hardlimit", 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "purged_snapdirs", 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "recovery_deletes", 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "sortbitwise" 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_version": 5, 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_snapshot": "", 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_max": 3, 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "max_osd": 8, 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "require_min_compat_client": "luminous", 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "min_compat_client": "luminous", 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "require_osd_release": "squid", 2024-05-08T08:27:09.405 INFO:teuthology.orchestra.run.smithi031.stdout: "allow_crimson": false, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "pools": [ 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 1, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "rbd", 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:55.134472+0000", 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:27:09.406 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 8, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 8, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 8, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 8, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 8, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:27:09.407 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "16", 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:27:09.408 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:27:09.409 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:27:09.410 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "error_message": "pool 1 has some PGs where all OSDs are with primary_affinity 0 (at least pg 1.4), read balance score may not be reliable", 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 0, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 0 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 2, 2024-05-08T08:27:09.411 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": ".mgr", 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:24:56.465515+0000", 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 1, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool", 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 1, 2024-05-08T08:27:09.412 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 1, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 1, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 1, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 1, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "17", 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:27:09.413 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 0, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 0, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:27:09.414 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:27:09.415 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "options": { 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_max": 32, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_min": 1 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.418 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "mgr": {} 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "score_type": "Fair distribution", 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 6.059999942779541, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 6.059999942779541, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "optimal_score": 0.33000001311302185, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_acting": 2, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "raw_score_stable": 2, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity_weighted": 0.69752693176269531, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity": 0.93000000715255737, 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: "average_primary_affinity_weighted": 0.69999998807907104 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.419 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_name": "pool_client.0", 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "create_time": "2024-05-08T08:25:03.054989+0000", 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags": 8193, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "type": 1, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 2, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "min_size": 1, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_rule": 0, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_count": 0, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_target": 0, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T08:27:09.420 INFO:teuthology.orchestra.run.smithi031.stdout: "object_hash": 2, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_autoscale_mode": "off", 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num": 32, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num": 32, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_placement_num_target": 32, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_target": 32, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_num_pending": 32, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "last_pg_merge_meta": { 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "source_pgid": "0.0", 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "ready_epoch": 0, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_started": 0, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "last_epoch_clean": 0, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "source_version": "0'0", 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "target_version": "0'0" 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.421 INFO:teuthology.orchestra.run.smithi031.stdout: "last_change": "133", 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend": "0", 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "auid": 0, 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_mode": "selfmanaged", 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_seq": 39, 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "snap_epoch": 133, 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "pool_snaps": [], 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps": "[]", 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_bytes": 0, 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "quota_max_objects": 0, 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tiers": [], 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "tier_of": -1, 2024-05-08T08:27:09.422 INFO:teuthology.orchestra.run.smithi031.stdout: "read_tier": -1, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "write_tier": -1, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_mode": "none", 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_bytes": 0, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "target_max_objects": 0, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_flush_age": 0, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "cache_min_evict_age": 0, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profile": "", 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_params": { 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "none" 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_period": 0, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_count": 0, 2024-05-08T08:27:09.423 INFO:teuthology.orchestra.run.smithi031.stdout: "use_gmt_hitset": true, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min_read_recency_for_promote": 0, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "min_write_recency_for_promote": 0, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "hit_set_search_last_n": 0, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "grade_table": [], 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "stripe_width": 0, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "expected_num_objects": 0, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "fast_read": false, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "options": {}, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "application_metadata": { 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "rbd": {} 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "read_balance": { 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "error_message": "pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.2), read balance score may not be reliable", 2024-05-08T08:27:09.424 INFO:teuthology.orchestra.run.smithi031.stdout: "score_acting": 0, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "score_stable": 0 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14afea42-356a-4163-ac3f-2e270e807917", 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:27:09.425 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 130, 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6800", 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6801", 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.426 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6802", 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6803", 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.427 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6806", 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6807", 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.428 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6804", 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6805", 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 727789107 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.429 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6801/727789107", 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6803/727789107", 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6807/727789107", 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6805/727789107", 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "db77a923-8f49-4b47-b752-00b475917a0b", 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:27:09.430 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 13, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 30, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 54, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 130, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 32, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6816", 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.431 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6817", 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6818", 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.432 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6819", 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6822", 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.433 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6823", 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6820", 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.434 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6821", 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2489759853 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6817/2489759853", 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6819/2489759853", 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6823/2489759853", 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6821/2489759853", 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.435 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "3d508691-8e40-4592-82d0-eaf4c4c7d7dd", 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 0.582733154296875, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0.5802154541015625, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 110, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.436 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6808", 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6809", 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.437 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6810", 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6811", 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.438 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6814", 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6815", 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.439 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6812", 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6813", 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 2736372115 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6809/2736372115", 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6811/2736372115", 2024-05-08T08:27:09.440 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6815/2736372115", 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6813/2736372115", 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "2545300a-428a-41ed-b19a-3a0cb9e5e755", 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0, 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:27:09.441 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 124, 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6824", 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.442 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6825", 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6826", 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6827", 2024-05-08T08:27:09.443 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6830", 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6831", 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.444 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6828", 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.31:6829", 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 573482428 2024-05-08T08:27:09.445 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.31:6825/573482428", 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.31:6827/573482428", 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.31:6831/573482428", 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.31:6829/573482428", 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "886aeed5-4495-4064-8579-050d38a4ad1b", 2024-05-08T08:27:09.446 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 0, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 91, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 110, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6801", 2024-05-08T08:27:09.447 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6803", 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6805", 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.448 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6807", 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6813", 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.449 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6815", 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6809", 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.450 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6811", 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3424463091 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6803/3424463091", 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6807/3424463091", 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6815/3424463091", 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6811/3424463091", 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: "exists" 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.451 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "4d15cef0-9ee9-4069-a6c4-a650e873ee03", 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 0, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 128, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.452 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6816", 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6818", 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.453 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6820", 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6822", 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.454 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6828", 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6830", 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.455 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6824", 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6826", 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 3709461555 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6818/3709461555", 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6822/3709461555", 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6830/3709461555", 2024-05-08T08:27:09.456 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6826/3709461555", 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "910ac664-9573-4851-9e2b-89ac6015fdc1", 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:27:09.457 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 131, 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6800", 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.458 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6802", 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6804", 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.459 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6806", 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6812", 2024-05-08T08:27:09.460 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6814", 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6808", 2024-05-08T08:27:09.461 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6810", 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 405561966 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6802/405561966", 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6806/405561966", 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6814/405561966", 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6810/405561966", 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.462 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "uuid": "14cc2d94-48e4-47c9-896a-9be9281ef0c5", 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "up": 1, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "in": 1, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "weight": 1, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_affinity": 1, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_begin": 0, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "last_clean_end": 0, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "up_from": 13, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "up_thru": 125, 2024-05-08T08:27:09.463 INFO:teuthology.orchestra.run.smithi031.stdout: "down_at": 0, 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "lost_at": 0, 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addrs": { 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6817", 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6819", 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.464 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addrs": { 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6821", 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6823", 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.465 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addrs": { 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6829", 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6831", 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.466 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addrs": { 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "addrvec": [ 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v2", 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6825", 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "type": "v1", 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "addr": "172.21.15.149:6827", 2024-05-08T08:27:09.467 INFO:teuthology.orchestra.run.smithi031.stdout: "nonce": 570236217 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "public_addr": "172.21.15.149:6819/570236217", 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "cluster_addr": "172.21.15.149:6823/570236217", 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_back_addr": "172.21.15.149:6831/570236217", 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "heartbeat_front_addr": "172.21.15.149:6827/570236217", 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "state": [ 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "exists", 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "up" 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: "osd_xinfo": [ 2024-05-08T08:27:09.468 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 0, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.572981+0000", 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 1, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.469 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.626543+0000", 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 32 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 2, 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.638878+0000", 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:27:09.470 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 3, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.599293+0000", 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 4, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.471 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.565676+0000", 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 110 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 5, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.552124+0000", 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:27:09.472 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 6, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.531428+0000", 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "osd": 7, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "down_stamp": "0.000000", 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_probability": 0, 2024-05-08T08:27:09.473 INFO:teuthology.orchestra.run.smithi031.stdout: "laggy_interval": 0, 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "features": 4540701547738038271, 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "old_weight": 0, 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "last_purged_snaps_scrub": "2024-05-08T08:24:51.658391+0000", 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "dead_epoch": 0 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap": [ 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.3", 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: 2, 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: 1 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.474 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.a", 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: 3, 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: 4 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_items": [ 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.2", 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 3, 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 4 2024-05-08T08:27:09.475 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.8", 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 3, 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 4 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: "mappings": [ 2024-05-08T08:27:09.476 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: "from": 0, 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: "to": 4 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_upmap_primaries": [], 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: "pg_temp": [ 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "1.5", 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: 6, 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: 5 2024-05-08T08:27:09.477 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.2", 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: 3 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.8", 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: 1, 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: 3 2024-05-08T08:27:09.478 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.e", 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: 7, 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: 6 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: "pgid": "3.17", 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: "osds": [ 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: 5, 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: 6 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.479 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "primary_temp": [], 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "blocklist": {}, 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "range_blocklist": {}, 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "erasure_code_profiles": { 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "default": { 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "crush-failure-domain": "osd", 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "k": "2", 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "m": "1", 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "plugin": "jerasure", 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "technique": "reed_sol_van" 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: "removed_snaps_queue": [ 2024-05-08T08:27:09.480 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 7, 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 14 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "new_removed_snaps": [ 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "pool": 3, 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "snaps": [ 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:09.481 INFO:teuthology.orchestra.run.smithi031.stdout: "begin": 7, 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "length": 1 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: ], 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "new_purged_snaps": [], 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "crush_node_flags": {}, 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "device_class_flags": {}, 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode": { 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_enabled": false, 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_bucket_count": 0, 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "degraded_stretch_mode": 0, 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "recovering_stretch_mode": 0, 2024-05-08T08:27:09.482 INFO:teuthology.orchestra.run.smithi031.stdout: "stretch_mode_bucket": 0 2024-05-08T08:27:09.483 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:09.483 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:09.485 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:27:09.486 INFO:tasks.thrashosds.thrasher:j is {'epoch': 133, 'fsid': '1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf', 'created': '2024-05-08T08:24:44.143941+0000', 'modified': '2024-05-08T08:27:09.082681+0000', 'last_up_change': '2024-05-08T08:26:46.068407+0000', 'last_in_change': '2024-05-08T08:27:04.049694+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 3, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-05-08T08:24:55.134472+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 1 has some PGs where all OSDs are with primary_affinity 0 (at least pg 1.4), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-05-08T08:24:56.465515+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.059999942779541, 'score_stable': 6.059999942779541, 'optimal_score': 0.33000001311302185, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.6975269317626953, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.699999988079071}}, {'pool': 3, 'pool_name': 'pool_client.0', 'create_time': '2024-05-08T08:25:03.054989+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '133', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 39, 'snap_epoch': 133, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'error_message': 'pool 3 has some PGs where all OSDs are with primary_affinity 0 (at least pg 3.2), read balance score may not be reliable', 'score_acting': 0, 'score_stable': 0}}], 'osds': [{'osd': 0, 'uuid': '14afea42-356a-4163-ac3f-2e270e807917', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 130, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6800', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6801', 'nonce': 727789107}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6802', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6803', 'nonce': 727789107}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6806', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6807', 'nonce': 727789107}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6804', 'nonce': 727789107}, {'type': 'v1', 'addr': '172.21.15.31:6805', 'nonce': 727789107}]}, 'public_addr': '172.21.15.31:6801/727789107', 'cluster_addr': '172.21.15.31:6803/727789107', 'heartbeat_back_addr': '172.21.15.31:6807/727789107', 'heartbeat_front_addr': '172.21.15.31:6805/727789107', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'db77a923-8f49-4b47-b752-00b475917a0b', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 30, 'up_from': 54, 'up_thru': 130, 'down_at': 32, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6816', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6817', 'nonce': 2489759853}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6818', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6819', 'nonce': 2489759853}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6822', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6823', 'nonce': 2489759853}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6820', 'nonce': 2489759853}, {'type': 'v1', 'addr': '172.21.15.31:6821', 'nonce': 2489759853}]}, 'public_addr': '172.21.15.31:6817/2489759853', 'cluster_addr': '172.21.15.31:6819/2489759853', 'heartbeat_back_addr': '172.21.15.31:6823/2489759853', 'heartbeat_front_addr': '172.21.15.31:6821/2489759853', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '3d508691-8e40-4592-82d0-eaf4c4c7d7dd', 'up': 1, 'in': 1, 'weight': 0.582733154296875, 'primary_affinity': 0.5802154541015625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 110, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6808', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6809', 'nonce': 2736372115}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6810', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6811', 'nonce': 2736372115}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6814', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6815', 'nonce': 2736372115}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6812', 'nonce': 2736372115}, {'type': 'v1', 'addr': '172.21.15.31:6813', 'nonce': 2736372115}]}, 'public_addr': '172.21.15.31:6809/2736372115', 'cluster_addr': '172.21.15.31:6811/2736372115', 'heartbeat_back_addr': '172.21.15.31:6815/2736372115', 'heartbeat_front_addr': '172.21.15.31:6813/2736372115', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2545300a-428a-41ed-b19a-3a0cb9e5e755', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 124, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6824', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6825', 'nonce': 573482428}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6826', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6827', 'nonce': 573482428}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6830', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6831', 'nonce': 573482428}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6828', 'nonce': 573482428}, {'type': 'v1', 'addr': '172.21.15.31:6829', 'nonce': 573482428}]}, 'public_addr': '172.21.15.31:6825/573482428', 'cluster_addr': '172.21.15.31:6827/573482428', 'heartbeat_back_addr': '172.21.15.31:6831/573482428', 'heartbeat_front_addr': '172.21.15.31:6829/573482428', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '886aeed5-4495-4064-8579-050d38a4ad1b', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 91, 'down_at': 110, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6801', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6803', 'nonce': 3424463091}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6805', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6807', 'nonce': 3424463091}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6813', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6815', 'nonce': 3424463091}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6809', 'nonce': 3424463091}, {'type': 'v1', 'addr': '172.21.15.149:6811', 'nonce': 3424463091}]}, 'public_addr': '172.21.15.149:6803/3424463091', 'cluster_addr': '172.21.15.149:6807/3424463091', 'heartbeat_back_addr': '172.21.15.149:6815/3424463091', 'heartbeat_front_addr': '172.21.15.149:6811/3424463091', 'state': ['exists']}, {'osd': 5, 'uuid': '4d15cef0-9ee9-4069-a6c4-a650e873ee03', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 128, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6816', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6818', 'nonce': 3709461555}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6820', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6822', 'nonce': 3709461555}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6828', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6830', 'nonce': 3709461555}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6824', 'nonce': 3709461555}, {'type': 'v1', 'addr': '172.21.15.149:6826', 'nonce': 3709461555}]}, 'public_addr': '172.21.15.149:6818/3709461555', 'cluster_addr': '172.21.15.149:6822/3709461555', 'heartbeat_back_addr': '172.21.15.149:6830/3709461555', 'heartbeat_front_addr': '172.21.15.149:6826/3709461555', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '910ac664-9573-4851-9e2b-89ac6015fdc1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 131, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6800', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6802', 'nonce': 405561966}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6804', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6806', 'nonce': 405561966}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6812', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6814', 'nonce': 405561966}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6808', 'nonce': 405561966}, {'type': 'v1', 'addr': '172.21.15.149:6810', 'nonce': 405561966}]}, 'public_addr': '172.21.15.149:6802/405561966', 'cluster_addr': '172.21.15.149:6806/405561966', 'heartbeat_back_addr': '172.21.15.149:6814/405561966', 'heartbeat_front_addr': '172.21.15.149:6810/405561966', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '14cc2d94-48e4-47c9-896a-9be9281ef0c5', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 125, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6817', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6819', 'nonce': 570236217}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6821', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6823', 'nonce': 570236217}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6829', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6831', 'nonce': 570236217}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.149:6825', 'nonce': 570236217}, {'type': 'v1', 'addr': '172.21.15.149:6827', 'nonce': 570236217}]}, 'public_addr': '172.21.15.149:6819/570236217', 'cluster_addr': '172.21.15.149:6823/570236217', 'heartbeat_back_addr': '172.21.15.149:6831/570236217', 'heartbeat_front_addr': '172.21.15.149:6827/570236217', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.572981+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.626543+0000', 'dead_epoch': 32}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.638878+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.599293+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.565676+0000', 'dead_epoch': 110}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.552124+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.531428+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T08:24:51.658391+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.3', 'osds': [2, 1]}, {'pgid': '3.a', 'osds': [3, 4]}], 'pg_upmap_items': [{'pgid': '3.2', 'mappings': [{'from': 3, 'to': 4}]}, {'pgid': '3.8', 'mappings': [{'from': 3, 'to': 4}]}, {'pgid': '3.17', 'mappings': [{'from': 0, 'to': 4}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '1.5', 'osds': [6, 5]}, {'pgid': '3.2', 'osds': [5, 3]}, {'pgid': '3.8', 'osds': [1, 3]}, {'pgid': '3.e', 'osds': [7, 6]}, {'pgid': '3.17', 'osds': [5, 6]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 3, 'snaps': [{'begin': 7, 'length': 14}]}], 'new_removed_snaps': [{'pool': 3, 'snaps': [{'begin': 7, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T08:27:09.486 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:27:09.487 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:09.491+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:09.584 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:09.592+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:09.592 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:09.592 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:09.601 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:27:09.685 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:09.692+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:09.707 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:09.707 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:09.716 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:27:09.800 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:09.807+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:09.826 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:09.830 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:27:09.838 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-05-08T08:27:09.089853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":338846911,"num_objects":600,"num_object_clones":132,"num_object_copies":1200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":274,"num_objects_misplaced":68,"num_objects_unfound":0,"num_objects_dirty":600,"num_whiteouts":400,"num_read":156690,"num_read_kb":860244,"num_write":21722,"num_write_kb":474445,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1765,"num_bytes_recovered":64894736,"num_keys_recovered":92,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17879,"ondisk_log_size":17879,"up":66,"acting":70,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":407352,"kb_used_data":27000,"kb_used_omap":24,"kb_used_meta":378663,"kb_avail":754567368,"statfs":{"total":773094113280,"available":772676984832,"internally_reserved":0,"allocated":27648000,"data_stored":169240966,"data_compressed":4027577,"data_compressed_allocated":12820480,"data_compressed_original":161695367,"omap_allocated":24704,"internal_metadata":387751808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-46154490,"num_objects":-50,"num_object_clones":-15,"num_object_copies":-100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-103,"num_objects_misplaced":-11,"num_objects_unfound":0,"num_objects_dirty":-50,"num_whiteouts":-13,"num_read":862,"num_read_kb":728,"num_write":90,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":2496710,"num_keys_recovered":0,"num_objects_omap":-2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.036347"},"pg_stats":[{"pgid":"3.1f","version":"126'736","reported_seq":5403,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.058053+0000","last_change":"2024-05-08T08:27:04.062498+0000","last_active":"2024-05-08T08:27:05.058053+0000","last_peered":"2024-05-08T08:27:05.058053+0000","last_clean":"2024-05-08T08:27:05.058053+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:05.058053+0000","last_undegraded":"2024-05-08T08:27:05.058053+0000","last_fullsized":"2024-05-08T08:27:05.058053+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":736,"log_dups_size":0,"ondisk_log_size":736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:06:43.565130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001928562,"stat_sum":{"num_bytes":15024793,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":11,"num_read":8489,"num_read_kb":28711,"num_write":725,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.1e","version":"126'772","reported_seq":3656,"reported_epoch":127,"state":"active+clean","last_fresh":"2024-05-08T08:27:03.064428+0000","last_change":"2024-05-08T08:27:03.065361+0000","last_active":"2024-05-08T08:27:03.064428+0000","last_peered":"2024-05-08T08:27:03.064428+0000","last_clean":"2024-05-08T08:27:03.064428+0000","last_became_active":"2024-05-08T08:27:02.040787+0000","last_became_peered":"2024-05-08T08:27:02.040787+0000","last_unstale":"2024-05-08T08:27:03.064428+0000","last_undegraded":"2024-05-08T08:27:03.064428+0000","last_fullsized":"2024-05-08T08:27:03.064428+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:29:38.740387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002984928,"stat_sum":{"num_bytes":7168549,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":17,"num_read":5248,"num_read_kb":27096,"num_write":1183,"num_write_kb":17969,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":121,"num_bytes_recovered":6658903,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"a","length":"b"}]},{"pgid":"3.1d","version":"127'548","reported_seq":2152,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.058277+0000","last_change":"2024-05-08T08:26:47.078032+0000","last_active":"2024-05-08T08:27:05.058277+0000","last_peered":"2024-05-08T08:27:05.058277+0000","last_clean":"2024-05-08T08:26:42.060701+0000","last_became_active":"2024-05-08T08:26:47.078032+0000","last_became_peered":"2024-05-08T08:26:47.078032+0000","last_unstale":"2024-05-08T08:27:05.058277+0000","last_undegraded":"2024-05-08T08:26:47.077432+0000","last_fullsized":"2024-05-08T08:26:47.077337+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":548,"log_dups_size":0,"ondisk_log_size":548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:03:05.221339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053686100000000002,"stat_sum":{"num_bytes":1010804,"num_objects":18,"num_object_clones":7,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":11,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":18}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"127'773","reported_seq":7754,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.057757+0000","last_change":"2024-05-08T08:26:47.077795+0000","last_active":"2024-05-08T08:27:05.057757+0000","last_peered":"2024-05-08T08:27:05.057757+0000","last_clean":"2024-05-08T08:26:44.076463+0000","last_became_active":"2024-05-08T08:26:47.077795+0000","last_became_peered":"2024-05-08T08:26:47.077795+0000","last_unstale":"2024-05-08T08:27:05.057757+0000","last_undegraded":"2024-05-08T08:26:47.077423+0000","last_fullsized":"2024-05-08T08:26:47.077332+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":773,"log_dups_size":0,"ondisk_log_size":773,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:09:56.661842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048139500000000002,"stat_sum":{"num_bytes":50483669,"num_objects":22,"num_object_clones":6,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":13,"num_read":9725,"num_read_kb":74667,"num_write":983,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":22}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"117'1071","reported_seq":2612,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.058391+0000","last_change":"2024-05-08T08:26:47.087640+0000","last_active":"2024-05-08T08:27:05.058391+0000","last_peered":"2024-05-08T08:27:05.058391+0000","last_clean":"2024-05-08T08:26:45.111492+0000","last_became_active":"2024-05-08T08:26:47.087640+0000","last_became_peered":"2024-05-08T08:26:47.087640+0000","last_unstale":"2024-05-08T08:27:05.058391+0000","last_undegraded":"2024-05-08T08:26:47.080713+0000","last_fullsized":"2024-05-08T08:26:47.080627+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1071,"log_dups_size":0,"ondisk_log_size":1071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":11,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:23:33.083798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.029322139000000001,"stat_sum":{"num_bytes":18110548,"num_objects":31,"num_object_clones":8,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":21,"num_read":6159,"num_read_kb":12596,"num_write":1691,"num_write_kb":13435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":31}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"14","length":"1"}]},{"pgid":"3.1a","version":"131'453","reported_seq":12562,"reported_epoch":132,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:08.091250+0000","last_change":"2024-05-08T08:26:47.082303+0000","last_active":"2024-05-08T08:27:08.091250+0000","last_peered":"2024-05-08T08:27:08.091250+0000","last_clean":"2024-05-08T08:26:42.132746+0000","last_became_active":"2024-05-08T08:26:47.082303+0000","last_became_peered":"2024-05-08T08:26:47.082303+0000","last_unstale":"2024-05-08T08:27:08.091250+0000","last_undegraded":"2024-05-08T08:26:47.080005+0000","last_fullsized":"2024-05-08T08:26:47.079902+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":453,"log_dups_size":0,"ondisk_log_size":453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":13,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:09:19.021734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052678600000000001,"stat_sum":{"num_bytes":1210009,"num_objects":23,"num_object_clones":6,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":16,"num_read":12101,"num_read_kb":37140,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":23}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"127'429","reported_seq":2266,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-08T08:27:08.091077+0000","last_change":"2024-05-08T08:27:07.083095+0000","last_active":"2024-05-08T08:27:08.091077+0000","last_peered":"2024-05-08T08:27:08.091077+0000","last_clean":"2024-05-08T08:27:08.091077+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:08.091077+0000","last_undegraded":"2024-05-08T08:27:08.091077+0000","last_fullsized":"2024-05-08T08:27:08.091077+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:52.730372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0021263419999999998,"stat_sum":{"num_bytes":127019,"num_objects":10,"num_object_clones":1,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":7,"num_read":2515,"num_read_kb":23482,"num_write":466,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"d"}]},{"pgid":"3.18","version":"124'406","reported_seq":12713,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.057959+0000","last_change":"2024-05-08T08:27:04.062777+0000","last_active":"2024-05-08T08:27:05.057959+0000","last_peered":"2024-05-08T08:27:05.057959+0000","last_clean":"2024-05-08T08:27:05.057959+0000","last_became_active":"2024-05-08T08:26:28.398428+0000","last_became_peered":"2024-05-08T08:26:28.398428+0000","last_unstale":"2024-05-08T08:27:05.057959+0000","last_undegraded":"2024-05-08T08:27:05.057959+0000","last_fullsized":"2024-05-08T08:27:05.057959+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":406,"log_dups_size":0,"ondisk_log_size":406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:54.331175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0018169569999999999,"stat_sum":{"num_bytes":1869337,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":12647,"num_read_kb":31472,"num_write":392,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.17","version":"128'763","reported_seq":2321,"reported_epoch":129,"state":"active+clean+remapped","last_fresh":"2024-05-08T08:27:05.058059+0000","last_change":"2024-05-08T08:27:04.062420+0000","last_active":"2024-05-08T08:27:05.058059+0000","last_peered":"2024-05-08T08:27:05.058059+0000","last_clean":"2024-05-08T08:27:05.058059+0000","last_became_active":"2024-05-08T08:27:02.044049+0000","last_became_peered":"2024-05-08T08:27:02.044049+0000","last_unstale":"2024-05-08T08:27:05.058059+0000","last_undegraded":"2024-05-08T08:27:05.058059+0000","last_fullsized":"2024-05-08T08:27:05.058059+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":763,"log_dups_size":0,"ondisk_log_size":763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:26:47.479145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026952040000000001,"stat_sum":{"num_bytes":262144,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":12,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":10,"num_read":4146,"num_read_kb":7810,"num_write":1064,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5,6","objects":12}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.16","version":"127'360","reported_seq":1962,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.104970+0000","last_change":"2024-05-08T08:27:04.058925+0000","last_active":"2024-05-08T08:27:05.104970+0000","last_peered":"2024-05-08T08:27:05.104970+0000","last_clean":"2024-05-08T08:27:05.104970+0000","last_became_active":"2024-05-08T08:26:04.253934+0000","last_became_peered":"2024-05-08T08:26:04.253934+0000","last_unstale":"2024-05-08T08:27:05.104970+0000","last_undegraded":"2024-05-08T08:27:05.104970+0000","last_fullsized":"2024-05-08T08:27:05.104970+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":360,"log_dups_size":0,"ondisk_log_size":360,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:19:53.369671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069037799999999996,"stat_sum":{"num_bytes":208709,"num_objects":19,"num_object_clones":3,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":13,"num_read":2210,"num_read_kb":14468,"num_write":343,"num_write_kb":11772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.15","version":"128'423","reported_seq":4105,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.105017+0000","last_change":"2024-05-08T08:27:04.063948+0000","last_active":"2024-05-08T08:27:05.105017+0000","last_peered":"2024-05-08T08:27:05.105017+0000","last_clean":"2024-05-08T08:27:05.105017+0000","last_became_active":"2024-05-08T08:27:02.043793+0000","last_became_peered":"2024-05-08T08:27:02.043793+0000","last_unstale":"2024-05-08T08:27:05.105017+0000","last_undegraded":"2024-05-08T08:27:05.105017+0000","last_fullsized":"2024-05-08T08:27:05.105017+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":423,"log_dups_size":0,"ondisk_log_size":423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:08.207051+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0055083780000000004,"stat_sum":{"num_bytes":8388608,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":425,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.14","version":"122'487","reported_seq":1398,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.104944+0000","last_change":"2024-05-08T08:26:47.080574+0000","last_active":"2024-05-08T08:27:05.104944+0000","last_peered":"2024-05-08T08:27:05.104944+0000","last_clean":"2024-05-08T08:26:38.140049+0000","last_became_active":"2024-05-08T08:26:47.080574+0000","last_became_peered":"2024-05-08T08:26:47.080574+0000","last_unstale":"2024-05-08T08:27:05.104944+0000","last_undegraded":"2024-05-08T08:26:47.079836+0000","last_fullsized":"2024-05-08T08:26:47.079750+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":487,"log_dups_size":0,"ondisk_log_size":487,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:18.273611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065286900000000004,"stat_sum":{"num_bytes":10971786,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":16,"num_read":1585,"num_read_kb":8807,"num_write":650,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":28}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.058369+0000","last_change":"2024-05-08T08:27:05.057403+0000","last_active":"2024-05-08T08:27:05.058369+0000","last_peered":"2024-05-08T08:27:05.058369+0000","last_clean":"2024-05-08T08:27:05.058369+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:05.058369+0000","last_undegraded":"2024-05-08T08:27:05.058369+0000","last_fullsized":"2024-05-08T08:27:05.058369+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:06.889029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"127'414","reported_seq":9097,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.058399+0000","last_change":"2024-05-08T08:27:04.063519+0000","last_active":"2024-05-08T08:27:05.058399+0000","last_peered":"2024-05-08T08:27:05.058399+0000","last_clean":"2024-05-08T08:27:05.058399+0000","last_became_active":"2024-05-08T08:26:28.402256+0000","last_became_peered":"2024-05-08T08:26:28.402256+0000","last_unstale":"2024-05-08T08:27:05.058399+0000","last_undegraded":"2024-05-08T08:27:05.058399+0000","last_fullsized":"2024-05-08T08:27:05.058399+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:11:27.645469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003085277,"stat_sum":{"num_bytes":807949,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":11,"num_read":8943,"num_read_kb":19928,"num_write":384,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.061534+0000","last_change":"2024-05-08T08:27:05.061031+0000","last_active":"2024-05-08T08:27:05.061534+0000","last_peered":"2024-05-08T08:27:05.061534+0000","last_clean":"2024-05-08T08:27:05.061534+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:05.061534+0000","last_undegraded":"2024-05-08T08:27:05.061534+0000","last_fullsized":"2024-05-08T08:27:05.061534+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:49:14.437090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"130'440","reported_seq":5392,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-08T08:27:08.091130+0000","last_change":"2024-05-08T08:27:07.085157+0000","last_active":"2024-05-08T08:27:08.091130+0000","last_peered":"2024-05-08T08:27:08.091130+0000","last_clean":"2024-05-08T08:27:08.091130+0000","last_became_active":"2024-05-08T08:26:01.241372+0000","last_became_peered":"2024-05-08T08:26:01.241372+0000","last_unstale":"2024-05-08T08:27:08.091130+0000","last_undegraded":"2024-05-08T08:27:08.091130+0000","last_fullsized":"2024-05-08T08:27:08.091130+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:06.135064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.004147258,"stat_sum":{"num_bytes":33525253,"num_objects":19,"num_object_clones":5,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":12,"num_read":5760,"num_read_kb":36923,"num_write":484,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":142,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"d"}]},{"pgid":"1.5","version":"0'0","reported_seq":11,"reported_epoch":124,"state":"active+clean+remapped","last_fresh":"2024-05-08T08:26:59.994380+0000","last_change":"2024-05-08T08:26:59.989406+0000","last_active":"2024-05-08T08:26:59.994380+0000","last_peered":"2024-05-08T08:26:59.994380+0000","last_clean":"2024-05-08T08:26:59.994380+0000","last_became_active":"2024-05-08T08:26:59.988946+0000","last_became_peered":"2024-05-08T08:26:59.988946+0000","last_unstale":"2024-05-08T08:26:59.994380+0000","last_undegraded":"2024-05-08T08:26:59.994380+0000","last_fullsized":"2024-05-08T08:26:59.994380+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:30:56.753630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.7","version":"126'823","reported_seq":7858,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.105126+0000","last_change":"2024-05-08T08:27:04.059310+0000","last_active":"2024-05-08T08:27:05.105126+0000","last_peered":"2024-05-08T08:27:05.105126+0000","last_clean":"2024-05-08T08:27:05.105126+0000","last_became_active":"2024-05-08T08:26:28.403209+0000","last_became_peered":"2024-05-08T08:26:28.403209+0000","last_unstale":"2024-05-08T08:27:05.105126+0000","last_undegraded":"2024-05-08T08:27:05.105126+0000","last_fullsized":"2024-05-08T08:27:05.105126+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":823,"log_dups_size":0,"ondisk_log_size":823,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:16:44.094238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010002469999999999,"stat_sum":{"num_bytes":16080040,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":14,"num_read":10607,"num_read_kb":75506,"num_write":1057,"num_write_kb":39605,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"1.4","version":"0'0","reported_seq":238,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.055379+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:27:05.055379+0000","last_peered":"2024-05-08T08:27:05.055379+0000","last_clean":"2024-05-08T08:27:05.055379+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:05.055379+0000","last_undegraded":"2024-05-08T08:27:05.055379+0000","last_fullsized":"2024-05-08T08:27:05.055379+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"127'474","reported_seq":3100,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.057856+0000","last_change":"2024-05-08T08:27:04.061484+0000","last_active":"2024-05-08T08:27:05.057856+0000","last_peered":"2024-05-08T08:27:05.057856+0000","last_clean":"2024-05-08T08:27:05.057856+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:05.057856+0000","last_undegraded":"2024-05-08T08:27:05.057856+0000","last_fullsized":"2024-05-08T08:27:05.057856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":474,"log_dups_size":0,"ondisk_log_size":474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:04:28.819115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092975700000000002,"stat_sum":{"num_bytes":22861746,"num_objects":23,"num_object_clones":3,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":15,"num_read":3558,"num_read_kb":39882,"num_write":444,"num_write_kb":20008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.1","version":"122'1224","reported_seq":2920,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.057319+0000","last_change":"2024-05-08T08:26:47.082374+0000","last_active":"2024-05-08T08:27:05.057319+0000","last_peered":"2024-05-08T08:27:05.057319+0000","last_clean":"2024-05-08T08:26:45.082054+0000","last_became_active":"2024-05-08T08:26:47.082374+0000","last_became_peered":"2024-05-08T08:26:47.082374+0000","last_unstale":"2024-05-08T08:27:05.057319+0000","last_undegraded":"2024-05-08T08:26:47.080501+0000","last_fullsized":"2024-05-08T08:26:47.080396+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1224,"log_dups_size":0,"ondisk_log_size":1224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:50:25.759148+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.007688678,"stat_sum":{"num_bytes":1483306,"num_objects":32,"num_object_clones":9,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":22,"num_read":6936,"num_read_kb":16297,"num_write":1877,"num_write_kb":9323,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":32}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":394,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.104863+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:27:05.104863+0000","last_peered":"2024-05-08T08:27:05.104863+0000","last_clean":"2024-05-08T08:27:05.104863+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:05.104863+0000","last_undegraded":"2024-05-08T08:27:05.104863+0000","last_fullsized":"2024-05-08T08:27:05.104863+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":183,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-08T08:27:08.091290+0000","last_change":"2024-05-08T08:27:05.059497+0000","last_active":"2024-05-08T08:27:08.091290+0000","last_peered":"2024-05-08T08:27:08.091290+0000","last_clean":"2024-05-08T08:27:08.091290+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:08.091290+0000","last_undegraded":"2024-05-08T08:27:08.091290+0000","last_fullsized":"2024-05-08T08:27:08.091290+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:50:56.853131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"131'356","reported_seq":1817,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-05-08T08:27:08.091090+0000","last_change":"2024-05-08T08:27:07.082431+0000","last_active":"2024-05-08T08:27:08.091090+0000","last_peered":"2024-05-08T08:27:08.091090+0000","last_clean":"2024-05-08T08:27:08.091090+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:08.091090+0000","last_undegraded":"2024-05-08T08:27:08.091090+0000","last_fullsized":"2024-05-08T08:27:08.091090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":356,"log_dups_size":0,"ondisk_log_size":356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:40:59.519072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001484874,"stat_sum":{"num_bytes":221200,"num_objects":18,"num_object_clones":3,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":14,"num_read":1052,"num_read_kb":15371,"num_write":375,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"d"}]},{"pgid":"1.2","version":"0'0","reported_seq":239,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.105106+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:27:05.105106+0000","last_peered":"2024-05-08T08:27:05.105106+0000","last_clean":"2024-05-08T08:27:05.105106+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:05.105106+0000","last_undegraded":"2024-05-08T08:27:05.105106+0000","last_fullsized":"2024-05-08T08:27:05.105106+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"127'766","reported_seq":6022,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.054892+0000","last_change":"2024-05-08T08:27:04.054529+0000","last_active":"2024-05-08T08:27:05.054892+0000","last_peered":"2024-05-08T08:27:05.054892+0000","last_clean":"2024-05-08T08:27:05.054892+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:05.054892+0000","last_undegraded":"2024-05-08T08:27:05.054892+0000","last_fullsized":"2024-05-08T08:27:05.054892+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":766,"log_dups_size":0,"ondisk_log_size":766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:35:50.695276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078727099999999998,"stat_sum":{"num_bytes":256000,"num_objects":11,"num_object_clones":1,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":8,"num_read":6632,"num_read_kb":37129,"num_write":840,"num_write_kb":25715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"1.1","version":"0'0","reported_seq":125,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.057526+0000","last_change":"2024-05-08T08:26:04.255319+0000","last_active":"2024-05-08T08:27:05.057526+0000","last_peered":"2024-05-08T08:27:05.057526+0000","last_clean":"2024-05-08T08:27:05.057526+0000","last_became_active":"2024-05-08T08:26:04.255074+0000","last_became_peered":"2024-05-08T08:26:04.255074+0000","last_unstale":"2024-05-08T08:27:05.057526+0000","last_undegraded":"2024-05-08T08:27:05.057526+0000","last_fullsized":"2024-05-08T08:27:05.057526+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:52:16.514474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"125'315","reported_seq":1874,"reported_epoch":129,"state":"active+clean+remapped","last_fresh":"2024-05-08T08:27:05.057559+0000","last_change":"2024-05-08T08:27:04.063174+0000","last_active":"2024-05-08T08:27:05.057559+0000","last_peered":"2024-05-08T08:27:05.057559+0000","last_clean":"2024-05-08T08:27:05.057559+0000","last_became_active":"2024-05-08T08:26:50.984261+0000","last_became_peered":"2024-05-08T08:26:50.984261+0000","last_unstale":"2024-05-08T08:27:05.057559+0000","last_undegraded":"2024-05-08T08:27:05.057559+0000","last_fullsized":"2024-05-08T08:27:05.057559+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:22.093975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0028249299999999998,"stat_sum":{"num_bytes":31124658,"num_objects":22,"num_object_clones":3,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":22,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":16,"num_read":1236,"num_read_kb":48448,"num_write":290,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[{"shards":"3,5","objects":22}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"1.0","version":"0'0","reported_seq":239,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.104883+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:27:05.104883+0000","last_peered":"2024-05-08T08:27:05.104883+0000","last_clean":"2024-05-08T08:27:05.104883+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:05.104883+0000","last_undegraded":"2024-05-08T08:27:05.104883+0000","last_fullsized":"2024-05-08T08:27:05.104883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"122'403","reported_seq":4705,"reported_epoch":132,"state":"active+clean+remapped","last_fresh":"2024-05-08T08:27:08.091108+0000","last_change":"2024-05-08T08:27:07.085185+0000","last_active":"2024-05-08T08:27:08.091108+0000","last_peered":"2024-05-08T08:27:08.091108+0000","last_clean":"2024-05-08T08:27:08.091108+0000","last_became_active":"2024-05-08T08:26:50.985998+0000","last_became_peered":"2024-05-08T08:26:50.985998+0000","last_unstale":"2024-05-08T08:27:08.091108+0000","last_undegraded":"2024-05-08T08:27:08.091108+0000","last_fullsized":"2024-05-08T08:27:08.091108+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":403,"log_dups_size":0,"ondisk_log_size":403,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:49.618938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0039481639999999997,"stat_sum":{"num_bytes":401548,"num_objects":20,"num_object_clones":4,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":20,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":13,"num_read":5732,"num_read_kb":26358,"num_write":413,"num_write_kb":20632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[{"shards":"1,3","objects":20}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"8","length":"d"}]},{"pgid":"3.9","version":"120'317","reported_seq":1536,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.054902+0000","last_change":"2024-05-08T08:26:47.082257+0000","last_active":"2024-05-08T08:27:05.054902+0000","last_peered":"2024-05-08T08:27:05.054902+0000","last_clean":"2024-05-08T08:26:44.076334+0000","last_became_active":"2024-05-08T08:26:47.082257+0000","last_became_peered":"2024-05-08T08:26:47.082257+0000","last_unstale":"2024-05-08T08:27:05.054902+0000","last_undegraded":"2024-05-08T08:26:47.079992+0000","last_fullsized":"2024-05-08T08:26:47.079884+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":317,"log_dups_size":0,"ondisk_log_size":317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:33:43.528132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075209599999999995,"stat_sum":{"num_bytes":923648,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":16,"num_read":2270,"num_read_kb":7349,"num_write":308,"num_write_kb":4376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[{"shards":"2","objects":24}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"127'646","reported_seq":1946,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.055166+0000","last_change":"2024-05-08T08:26:47.078343+0000","last_active":"2024-05-08T08:27:05.055166+0000","last_peered":"2024-05-08T08:27:05.055166+0000","last_clean":"2024-05-08T08:26:44.076316+0000","last_became_active":"2024-05-08T08:26:47.078343+0000","last_became_peered":"2024-05-08T08:26:47.078343+0000","last_unstale":"2024-05-08T08:27:05.055166+0000","last_undegraded":"2024-05-08T08:26:47.077699+0000","last_fullsized":"2024-05-08T08:26:47.077592+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":646,"log_dups_size":0,"ondisk_log_size":646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:59.985249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000707513,"stat_sum":{"num_bytes":21863716,"num_objects":26,"num_object_clones":6,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":15,"num_read":2243,"num_read_kb":18955,"num_write":818,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":107,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":26}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"127'361","reported_seq":2502,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.058271+0000","last_change":"2024-05-08T08:27:04.060868+0000","last_active":"2024-05-08T08:27:05.058271+0000","last_peered":"2024-05-08T08:27:05.058271+0000","last_clean":"2024-05-08T08:27:05.058271+0000","last_became_active":"2024-05-08T08:26:28.399037+0000","last_became_peered":"2024-05-08T08:26:28.399037+0000","last_unstale":"2024-05-08T08:27:05.058271+0000","last_undegraded":"2024-05-08T08:27:05.058271+0000","last_fullsized":"2024-05-08T08:27:05.058271+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":361,"log_dups_size":0,"ondisk_log_size":361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:30:54.917768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000796961,"stat_sum":{"num_bytes":194560,"num_objects":17,"num_object_clones":3,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":13,"num_read":1815,"num_read_kb":17245,"num_write":331,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.c","version":"128'372","reported_seq":4477,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.057749+0000","last_change":"2024-05-08T08:27:04.064616+0000","last_active":"2024-05-08T08:27:05.057749+0000","last_peered":"2024-05-08T08:27:05.057749+0000","last_clean":"2024-05-08T08:27:05.057749+0000","last_became_active":"2024-05-08T08:26:28.402309+0000","last_became_peered":"2024-05-08T08:26:28.402309+0000","last_unstale":"2024-05-08T08:27:05.057749+0000","last_undegraded":"2024-05-08T08:27:05.057749+0000","last_fullsized":"2024-05-08T08:27:05.057749+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:26.936280+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0045626659999999999,"stat_sum":{"num_bytes":8388608,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":412,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.d","version":"120'369","reported_seq":1791,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.105208+0000","last_change":"2024-05-08T08:27:04.060440+0000","last_active":"2024-05-08T08:27:05.105208+0000","last_peered":"2024-05-08T08:27:05.105208+0000","last_clean":"2024-05-08T08:27:05.105208+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:05.105208+0000","last_undegraded":"2024-05-08T08:27:05.105208+0000","last_fullsized":"2024-05-08T08:27:05.105208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":369,"log_dups_size":0,"ondisk_log_size":369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:34:06.387508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0017976310000000001,"stat_sum":{"num_bytes":407552,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":7,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.e","version":"117'657","reported_seq":4932,"reported_epoch":129,"state":"active+clean+remapped","last_fresh":"2024-05-08T08:27:05.105134+0000","last_change":"2024-05-08T08:27:04.059127+0000","last_active":"2024-05-08T08:27:05.105134+0000","last_peered":"2024-05-08T08:27:05.105134+0000","last_clean":"2024-05-08T08:27:05.105134+0000","last_became_active":"2024-05-08T08:26:59.987889+0000","last_became_peered":"2024-05-08T08:26:59.987889+0000","last_unstale":"2024-05-08T08:27:05.105134+0000","last_undegraded":"2024-05-08T08:27:05.105134+0000","last_fullsized":"2024-05-08T08:27:05.105134+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":657,"log_dups_size":0,"ondisk_log_size":657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:15:01.073558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086045200000000005,"stat_sum":{"num_bytes":13369344,"num_objects":14,"num_object_clones":1,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":14,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":12,"num_read":6437,"num_read_kb":47689,"num_write":651,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":149,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7,6],"avail_no_missing":["7","6"],"object_location_counts":[{"shards":"6,7","objects":14}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.f","version":"127'447","reported_seq":1757,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.105204+0000","last_change":"2024-05-08T08:26:47.080586+0000","last_active":"2024-05-08T08:27:05.105204+0000","last_peered":"2024-05-08T08:27:05.105204+0000","last_clean":"2024-05-08T08:26:45.071651+0000","last_became_active":"2024-05-08T08:26:47.080586+0000","last_became_peered":"2024-05-08T08:26:47.080586+0000","last_unstale":"2024-05-08T08:27:05.105204+0000","last_undegraded":"2024-05-08T08:26:47.079974+0000","last_fullsized":"2024-05-08T08:26:47.079892+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":447,"log_dups_size":0,"ondisk_log_size":447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:01:17.816686+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001300051,"stat_sum":{"num_bytes":3632667,"num_objects":16,"num_object_clones":4,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":10,"num_read":1854,"num_read_kb":7519,"num_write":552,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":16}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"126'668","reported_seq":2935,"reported_epoch":127,"state":"remapped+peering","last_fresh":"2024-05-08T08:27:03.060744+0000","last_change":"2024-05-08T08:27:05.057621+0000","last_active":"2024-05-08T08:27:05.057621+0000","last_peered":"2024-05-08T08:27:03.060744+0000","last_clean":"2024-05-08T08:27:03.060744+0000","last_became_active":"2024-05-08T08:27:02.044337+0000","last_became_peered":"2024-05-08T08:27:02.044337+0000","last_unstale":"2024-05-08T08:27:03.060744+0000","last_undegraded":"2024-05-08T08:27:03.060744+0000","last_fullsized":"2024-05-08T08:27:03.060744+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001799372,"stat_sum":{"num_bytes":506457,"num_objects":10,"num_object_clones":3,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":5,"num_read":3762,"num_read_kb":12483,"num_write":955,"num_write_kb":9114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":527085,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6],"up_primary":6,"acting_primary":0,"purged_snaps":[{"start":"a","length":"b"}]},{"pgid":"3.11","version":"126'450","reported_seq":1695,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.104915+0000","last_change":"2024-05-08T08:26:47.080215+0000","last_active":"2024-05-08T08:27:05.104915+0000","last_peered":"2024-05-08T08:27:05.104915+0000","last_clean":"2024-05-08T08:26:45.070286+0000","last_became_active":"2024-05-08T08:26:47.080215+0000","last_became_peered":"2024-05-08T08:26:47.080215+0000","last_unstale":"2024-05-08T08:27:05.104915+0000","last_undegraded":"2024-05-08T08:26:47.079752+0000","last_fullsized":"2024-05-08T08:26:47.079651+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:06:58.630944+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.00074078299999999996,"stat_sum":{"num_bytes":18999918,"num_objects":21,"num_object_clones":6,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":13,"num_read":1722,"num_read_kb":20588,"num_write":522,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":21}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"127'370","reported_seq":3368,"reported_epoch":129,"state":"active+clean","last_fresh":"2024-05-08T08:27:05.057655+0000","last_change":"2024-05-08T08:27:04.062226+0000","last_active":"2024-05-08T08:27:05.057655+0000","last_peered":"2024-05-08T08:27:05.057655+0000","last_clean":"2024-05-08T08:27:05.057655+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:05.057655+0000","last_undegraded":"2024-05-08T08:27:05.057655+0000","last_fullsized":"2024-05-08T08:27:05.057655+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":370,"log_dups_size":0,"ondisk_log_size":370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:43:56.702258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014325169999999999,"stat_sum":{"num_bytes":11528806,"num_objects":15,"num_object_clones":1,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":406,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"9","length":"c"}]},{"pgid":"3.13","version":"127'669","reported_seq":4246,"reported_epoch":129,"state":"active+undersized+degraded","last_fresh":"2024-05-08T08:27:05.104913+0000","last_change":"2024-05-08T08:26:47.080535+0000","last_active":"2024-05-08T08:27:05.104913+0000","last_peered":"2024-05-08T08:27:05.104913+0000","last_clean":"2024-05-08T08:26:45.070379+0000","last_became_active":"2024-05-08T08:26:47.080535+0000","last_became_peered":"2024-05-08T08:26:47.080535+0000","last_unstale":"2024-05-08T08:27:05.104913+0000","last_undegraded":"2024-05-08T08:26:47.079819+0000","last_fullsized":"2024-05-08T08:26:47.079692+0000","mapping_epoch":110,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":669,"log_dups_size":0,"ondisk_log_size":669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":12,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:17:36.038726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00067222699999999996,"stat_sum":{"num_bytes":36843592,"num_objects":33,"num_object_clones":11,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33,"num_whiteouts":16,"num_read":5278,"num_read_kb":19383,"num_write":976,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7],"acting":[7],"avail_no_missing":["7"],"object_location_counts":[{"shards":"7","objects":33}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":338256543,"num_objects":598,"num_object_clones":132,"num_object_copies":1196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":274,"num_objects_misplaced":68,"num_objects_unfound":0,"num_objects_dirty":598,"num_whiteouts":400,"num_read":156574,"num_read_kb":860144,"num_write":21563,"num_write_kb":472029,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1765,"num_bytes_recovered":64894736,"num_keys_recovered":92,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15011840,"data_stored":155701754,"data_compressed":3960693,"data_compressed_allocated":12292096,"data_compressed_original":153994887,"omap_allocated":9107,"internal_metadata":0},"log_size":17762,"ondisk_log_size":17762,"up":49,"acting":52,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":15,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574877,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53204,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94318636,"statfs":{"total":96636764160,"available":96582283264,"internally_reserved":0,"allocated":4063232,"data_stored":32038437,"data_compressed":610662,"data_compressed_allocated":2146304,"data_compressed_original":31055420,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41244,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330596,"statfs":{"total":96636764160,"available":96594530304,"internally_reserved":0,"allocated":1720320,"data_stored":2602700,"data_compressed":61946,"data_compressed_allocated":172032,"data_compressed_original":1855488,"omap_allocated":1592,"internal_metadata":40434120},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574877,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38812,"kb_used_data":3412,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333028,"statfs":{"total":96636764160,"available":96597020672,"internally_reserved":0,"allocated":3493888,"data_stored":23024136,"data_compressed":439407,"data_compressed_allocated":1568768,"data_compressed_original":22031391,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82304,"kb_used_data":6692,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94289536,"statfs":{"total":96636764160,"available":96552484864,"internally_reserved":0,"allocated":6852608,"data_stored":46766287,"data_compressed":1901375,"data_compressed_allocated":4554752,"data_compressed_original":45525494,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574878,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35192,"kb_used_data":4012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94336648,"statfs":{"total":96636764160,"available":96600727552,"internally_reserved":0,"allocated":4108288,"data_stored":34063486,"data_compressed":437110,"data_compressed_allocated":2240512,"data_compressed_original":33127263,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37020,"kb_used_data":1680,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334820,"statfs":{"total":96636764160,"available":96598855680,"internally_reserved":0,"allocated":1720320,"data_stored":1205137,"data_compressed":71788,"data_compressed_allocated":94208,"data_compressed_original":401408,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234002,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40992,"kb_used_data":2572,"kb_used_omap":13,"kb_used_meta":36978,"kb_avail":94330848,"statfs":{"total":96636764160,"available":96594788352,"internally_reserved":0,"allocated":2633728,"data_stored":11091914,"data_compressed":190516,"data_compressed_allocated":806912,"data_compressed_original":10162176,"omap_allocated":13550,"internal_metadata":37866258},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574877,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":78584,"kb_used_data":2984,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94293256,"statfs":{"total":96636764160,"available":96556294144,"internally_reserved":0,"allocated":3055616,"data_stored":18448869,"data_compressed":314773,"data_compressed_allocated":1236992,"data_compressed_original":17536727,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1523712,"data_stored":17711658,"data_compressed":314773,"data_compressed_allocated":1236992,"data_compressed_original":17536727,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":520192,"data_stored":3168636,"data_compressed":128575,"data_compressed_allocated":315392,"data_compressed_original":3035136,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":461642,"data_compressed":71788,"data_compressed_allocated":94208,"data_compressed_original":401408,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2564096,"data_stored":33319991,"data_compressed":437110,"data_compressed_allocated":2240512,"data_compressed_original":33127263,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5627904,"data_stored":46189123,"data_compressed":1901375,"data_compressed_allocated":4554752,"data_compressed_original":45525494,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1961984,"data_stored":22286925,"data_compressed":439407,"data_compressed_allocated":1568768,"data_compressed_original":22031391,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":176128,"data_stored":1859205,"data_compressed":61946,"data_compressed_allocated":172032,"data_compressed_original":1855488,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2461696,"data_stored":30704574,"data_compressed":605719,"data_compressed_allocated":2109440,"data_compressed_original":30481980,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:27:09.841 INFO:tasks.thrashosds.thrasher:Setting 3.15 to [1, 7] 2024-05-08T08:27:09.841 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.15', '1', '7'] 2024-05-08T08:27:09.841 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 3.15 1 7 2024-05-08T08:27:09.865 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:09.865 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:09.874 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:27:09.915 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:09.922+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:09.987 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:09.995+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:10.021 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:10.021 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:10.030 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:27:10.087 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:27:10.088 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:10.094+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:10.142 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:10.142 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:10.151 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:27:10.192 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:10.199+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:10.291 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:10.298+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:10.297 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:10.297 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:10.303 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T08:27:10.306 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:27:10.391 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:10.398+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:10.413 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:10.422 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:27:10.429 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:27:10.429 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:27:10.447 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:27:10.447 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:27:10.464 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:27:10.464 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:27:10.481 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:27:10.482 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:27:10.482 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:27:10.500 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:27:10.500 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:27:10.506 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:10.513+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:10.519 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:27:10.519 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:27:10.538 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:27:10.538 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:27:10.541 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:27:10.541 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: "size": 20, 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 600, 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [ 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4207.0:1 2.0 2:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 133.014265833, 2024-05-08T08:27:10.573 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.063839509000000003, 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4207", 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.149:0/3425589641", 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 1 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.574 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565606+0000", 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565608+0000", 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.302e-06 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565615+0000", 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.4370000000000001e-06 2024-05-08T08:27:10.575 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565636+0000", 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0577999999999998e-05 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565644+0000", 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.2779999999999993e-06 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565703+0000", 2024-05-08T08:27:10.576 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.8844000000000001e-05 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for peered", 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.565714+0000", 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1004e-05 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.620931+0000", 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.055217091000000003 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621084+0000", 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.000152817 2024-05-08T08:27:10.577 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "waiting for subops from 1", 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.621278+0000", 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.00019463600000000001 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "op_commit", 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.622604+0000", 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0013252559999999999 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629335+0000", 2024-05-08T08:27:10.578 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.0067316340000000002 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "commit_sent", 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629397+0000", 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.2135000000000001e-05 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:24:57.629446+0000", 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8497e-05 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.579 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69424 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 2883584~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.790160525999994, 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040548219000000003, 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69424 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.580 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789712+0000", 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000002e-07 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.581 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789715+0000", 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4959999999999999e-06 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789723+0000", 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.763e-06 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789728+0000", 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2909999999999998e-06 2024-05-08T08:27:10.582 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830163+0000", 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040434952000000003 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830203+0000", 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0482999999999998e-05 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830260+0000", 2024-05-08T08:27:10.583 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.6601000000000002e-05 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69425 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3112960~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.790075372000004, 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040591552000000003, 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.584 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69425 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789797+0000", 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.585 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789798+0000", 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.7400000000000003e-07 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789801+0000", 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.393e-06 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789810+0000", 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7949999999999996e-06 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.586 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789816+0000", 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9540000000000001e-06 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830288+0000", 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040472529 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830335+0000", 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.6353999999999997e-05 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.587 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830388+0000", 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3853000000000001e-05 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69426 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3342336~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.790047466000004, 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040673171000000001, 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.588 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69426 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.589 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789825+0000", 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.27e-07 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789827+0000", 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.193e-06 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789834+0000", 2024-05-08T08:27:10.590 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.0600000000000002e-06 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789840+0000", 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830414+0000", 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040574022000000001 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830450+0000", 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6009e-05 2024-05-08T08:27:10.591 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830498+0000", 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8220999999999998e-05 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69427 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3571712~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.790019064999996, 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040763890999999997, 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.592 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69427 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789853+0000", 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.593 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789854+0000", 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3600000000000004e-07 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789857+0000", 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.9220000000000001e-06 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789869+0000", 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.2207e-05 2024-05-08T08:27:10.594 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789875+0000", 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3300000000000004e-06 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830521+0000", 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040646175999999999 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830561+0000", 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9907000000000003e-05 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.595 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830617+0000", 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5813000000000002e-05 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69428 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 3801088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789934926000001, 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040799239000000001, 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.596 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69428 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789937+0000", 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.597 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789938+0000", 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 7.3799999999999996e-07 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789942+0000", 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5209999999999998e-06 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789953+0000", 2024-05-08T08:27:10.598 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.172e-05 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.789960+0000", 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.319e-06 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830647+0000", 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040687565000000002 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830686+0000", 2024-05-08T08:27:10.599 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8457999999999997e-05 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830737+0000", 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0918000000000003e-05 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69429 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4030464~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789854430999995, 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040824088000000001, 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.600 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69429 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.601 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790018+0000", 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.5799999999999999e-07 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790022+0000", 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2540000000000001e-06 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790033+0000", 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.0981999999999999e-05 2024-05-08T08:27:10.602 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790039+0000", 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9560000000000002e-06 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830761+0000", 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.04072191 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830796+0000", 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5549999999999997e-05 2024-05-08T08:27:10.603 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830842+0000", 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5778000000000001e-05 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69430 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4259840~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789821850999999, 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040910730999999999, 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.604 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69430 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790050+0000", 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.605 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790051+0000", 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6899999999999998e-07 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790053+0000", 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.888e-06 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790061+0000", 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1310000000000006e-06 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.606 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790070+0000", 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6950000000000006e-06 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830867+0000", 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040797630000000001 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830906+0000", 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8716e-05 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.607 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830961+0000", 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.5302e-05 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69431 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4489216~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789768659999993, 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040969653000000002, 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.608 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69431 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.609 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790104+0000", 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2099999999999997e-07 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790109+0000", 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1460000000000004e-06 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790114+0000", 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9080000000000003e-06 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.610 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790118+0000", 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4230000000000001e-06 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.830985+0000", 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040867677999999998 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831023+0000", 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7487999999999998e-05 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.611 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831073+0000", 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.0488999999999997e-05 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69432 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4718592~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789742681000007, 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041049254, 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.612 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69432 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.613 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790130+0000", 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.01e-07 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790132+0000", 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.711e-06 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790138+0000", 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.088e-06 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.614 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790141+0000", 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6969999999999999e-06 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831099+0000", 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957169000000002 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831133+0000", 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4792000000000002e-05 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.615 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831179+0000", 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5395999999999997e-05 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69433 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 4947968~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789687349000005, 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041112572999999999, 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.616 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69433 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.617 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790185+0000", 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2399999999999998e-07 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790188+0000", 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.0410000000000002e-06 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790197+0000", 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1529999999999997e-06 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.618 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790202+0000", 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1290000000000004e-06 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831206+0000", 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041003994000000002 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831243+0000", 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7452e-05 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.619 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831297+0000", 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.4280000000000002e-05 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69434 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5177344~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789537315999993, 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041062670000000003, 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.620 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69434 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790335+0000", 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.621 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790336+0000", 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.8899999999999999e-07 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790350+0000", 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.4290999999999999e-05 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790359+0000", 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.7669999999999992e-06 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.622 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790365+0000", 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.3260000000000001e-06 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831323+0000", 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040957876999999997 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831360+0000", 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7339999999999998e-05 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.623 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831398+0000", 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7379999999999998e-05 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69435 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5406720~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789487656999995, 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041106187000000002, 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.624 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69435 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.625 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790385+0000", 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.3200000000000005e-07 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790388+0000", 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.818e-06 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790395+0000", 2024-05-08T08:27:10.626 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6590000000000001e-06 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790410+0000", 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.5648e-05 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831421+0000", 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041010291999999997 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.627 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831455+0000", 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.4255000000000003e-05 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831491+0000", 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5982999999999998e-05 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69436 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5636096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789453088000002, 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041175124, 2024-05-08T08:27:10.628 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69436 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.629 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790419+0000", 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790420+0000", 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.89e-07 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790422+0000", 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.791e-06 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.630 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790429+0000", 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.6830000000000002e-06 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790434+0000", 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.2070000000000003e-06 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831519+0000", 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041084457999999997 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.631 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831556+0000", 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6986000000000001e-05 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831594+0000", 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.8510000000000002e-05 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69437 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 5865472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.632 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789431081000004, 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041258937000000002, 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69437 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.633 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790441+0000", 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790442+0000", 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.1600000000000002e-07 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790444+0000", 2024-05-08T08:27:10.634 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2010000000000002e-06 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790450+0000", 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.9039999999999997e-06 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790455+0000", 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8620000000000002e-06 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831619+0000", 2024-05-08T08:27:10.635 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041164693000000002 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831660+0000", 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.0902999999999998e-05 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831700+0000", 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9957999999999999e-05 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.636 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69438 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6094848~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789408107, 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041331442000000003, 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69438 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.637 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790464+0000", 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790465+0000", 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.638 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790467+0000", 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.2299999999999998e-06 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790473+0000", 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 6.0109999999999997e-06 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790477+0000", 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.5970000000000002e-06 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.639 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831725+0000", 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041247673999999998 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831760+0000", 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.506e-05 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831796+0000", 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.5502e-05 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.640 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69439 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 6324224~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.789378428999996, 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.041401766999999999, 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69439 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.641 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.790494+0000", 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791631+0000", 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.001137243 2024-05-08T08:27:10.642 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791636+0000", 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 5.1569999999999999e-06 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791646+0000", 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 9.8789999999999998e-06 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791658+0000", 2024-05-08T08:27:10.643 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.1413999999999999e-05 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831818+0000", 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040160639999999997 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831858+0000", 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9641000000000002e-05 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.831896+0000", 2024-05-08T08:27:10.644 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7793000000000002e-05 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69442 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7012352~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.788155850999999, 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040578492000000001, 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.645 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69442 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.646 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791716+0000", 2024-05-08T08:27:10.649 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791717+0000", 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5100000000000001e-07 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791718+0000", 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.362e-06 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.650 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791721+0000", 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.2040000000000002e-06 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791724+0000", 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.4760000000000001e-06 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832129+0000", 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040405231999999999 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.651 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832210+0000", 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.1391999999999996e-05 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832295+0000", 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.4574999999999996e-05 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69443 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7241728~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.652 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.788145357000005, 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040675411000000002, 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69443 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.653 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791727+0000", 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.0800000000000001e-07 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791728+0000", 2024-05-08T08:27:10.654 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.159e-06 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791733+0000", 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.8509999999999998e-06 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.791737+0000", 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.9009999999999996e-06 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.655 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832327+0000", 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040589821999999998 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832365+0000", 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7759999999999998e-05 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.832402+0000", 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.7710000000000003e-05 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.656 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "description": "osd_op(client.4497.0:69572 3.e 3:7380e385:::rbd_data.1191d80f8e36.0000000000000003:1e [read 7258112~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "initiated_at": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "age": 68.783026520999996, 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040499417000000003, 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "continuous": false, 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "type_data": { 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "flag_point": "started", 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "client_info": { 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "client": "client.4497", 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "tid": 69572 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "events": [ 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "initiated", 2024-05-08T08:27:10.657 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "header_read", 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "throttled", 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796846+0000", 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 2.5899999999999998e-07 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.658 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "all_read", 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796847+0000", 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 1.3379999999999999e-06 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "dispatched", 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796851+0000", 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 3.6150000000000001e-06 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "queued_for_pg", 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.796860+0000", 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 8.6479999999999992e-06 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.659 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "reached_pg", 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837249+0000", 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 0.040388986000000002 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "started", 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837298+0000", 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.9008999999999998e-05 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: }, 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: { 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "event": "done", 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "time": "2024-05-08T08:26:01.837345+0000", 2024-05-08T08:27:10.660 INFO:teuthology.orchestra.run.smithi149.stdout: "duration": 4.7562000000000003e-05 2024-05-08T08:27:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: } 2024-05-08T08:27:10.661 INFO:teuthology.orchestra.run.smithi149.stdout: ] 2024-05-08T08:27:10.661 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:10.662 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:27:10.663 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:10.670+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:10.763 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:10.771+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:10.772 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:10.772 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:10.773 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:10.773 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:10.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:27:10.866 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:10.873+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:10.931 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:10.931 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:10.932 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:10.940 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:27:10.982 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:10.989+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:11.066 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:11.074+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:11.090 INFO:teuthology.orchestra.run.smithi031.stderr:set 3.15 pg_upmap mapping to [1,7] 2024-05-08T08:27:11.095 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:11.095 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54357 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1028096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.941351186999995, 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058855941000000002, 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54357 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.096 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160254+0000", 2024-05-08T08:27:11.097 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9200000000000002e-07 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160257+0000", 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0999999999999998e-06 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160263+0000", 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1569999999999997e-06 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160268+0000", 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8779999999999997e-06 2024-05-08T08:27:11.098 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164225+0000", 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039576619999999998 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.164265+0000", 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9493999999999998e-05 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219110+0000", 2024-05-08T08:27:11.099 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.054845258000000001 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54358 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1037312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.941330086999997, 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058995199999999998, 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.100 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54358 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.101 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160275+0000", 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.2000000000000001e-07 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160278+0000", 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.21e-06 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160284+0000", 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5799999999999997e-06 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.102 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160289+0000", 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1379999999999998e-06 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219148+0000", 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058858599999999997 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219215+0000", 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7318999999999997e-05 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.103 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219270+0000", 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5033e-05 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54359 3.4 3:26c2a46c:::rbd_data.11914ad0077c.0000000000000002:1a [read 1046528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 78.941309429, 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.059150727, 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.104 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54359 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:11.105 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160296+0000", 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.1899999999999998e-07 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160298+0000", 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.075e-06 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.106 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5479999999999998e-06 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160310+0000", 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1100000000000002e-06 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219304+0000", 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.058993628999999999 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219376+0000", 2024-05-08T08:27:11.107 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2198000000000002e-05 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.219447+0000", 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0847999999999997e-05 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108422 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 491520~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107305552, 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032000842000000002, 2024-05-08T08:27:11.108 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108422 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.109 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994300+0000", 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9500000000000003e-07 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994307+0000", 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5130000000000002e-06 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.110 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994314+0000", 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2099999999999996e-06 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994320+0000", 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7760000000000003e-06 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026109+0000", 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031789339 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.111 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026194+0000", 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4841999999999998e-05 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026301+0000", 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000106667 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.112 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108428 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3145728~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107277943, 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032197602999999998, 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108428 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:27:11.113 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994327+0000", 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994328+0000", 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994330+0000", 2024-05-08T08:27:11.114 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5689999999999999e-06 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994338+0000", 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5830000000000001e-06 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994343+0000", 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9649999999999999e-06 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026333+0000", 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031990022 2024-05-08T08:27:11.115 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026419+0000", 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6278000000000003e-05 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026525+0000", 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010569 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108429 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3178496~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:11.116 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107254662999999, 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032394017999999997, 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108429 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.117 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994351+0000", 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0999999999999999e-07 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994353+0000", 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3829999999999999e-06 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.118 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994360+0000", 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6850000000000004e-06 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994365+0000", 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2680000000000002e-06 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026559+0000", 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032193927999999997 2024-05-08T08:27:11.119 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026647+0000", 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7882999999999999e-05 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026745+0000", 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7460999999999994e-05 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.120 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108430 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3211264~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107232695, 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032603935000000001, 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108430 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.121 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994373+0000", 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9999999999999998e-07 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.122 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994375+0000", 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3989999999999998e-06 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994383+0000", 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2869999999999999e-06 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994387+0000", 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8040000000000001e-06 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.123 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026777+0000", 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032389793 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026865+0000", 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7763999999999995e-05 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.026976+0000", 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000111488 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.124 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108431 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3244032~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107210287999997, 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032796189000000003, 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108431 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.125 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994395+0000", 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.4000000000000002e-07 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.126 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994398+0000", 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994404+0000", 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3969999999999999e-06 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994410+0000", 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2739999999999998e-06 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.127 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027010+0000", 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032600259999999999 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027094+0000", 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128999999999994e-05 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027191+0000", 2024-05-08T08:27:11.128 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7089000000000005e-05 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108444 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3162112~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107188418, 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032973852999999997, 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.129 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108444 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.130 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994417+0000", 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6300000000000001e-07 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994419+0000", 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2900000000000001e-06 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994426+0000", 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5080000000000002e-06 2024-05-08T08:27:11.131 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994431+0000", 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0200000000000002e-06 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027223+0000", 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032791703999999998 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027303+0000", 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0068999999999995e-05 2024-05-08T08:27:11.132 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027391+0000", 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7899000000000007e-05 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108445 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3194880~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107167394000001, 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033150011, 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.133 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108445 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.134 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994438+0000", 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2599999999999998e-07 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994441+0000", 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3700000000000002e-06 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.135 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994448+0000", 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9929999999999998e-06 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994452+0000", 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8199999999999996e-06 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027420+0000", 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032967989000000003 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.136 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027500+0000", 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9332000000000005e-05 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027588+0000", 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8080999999999999e-05 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108446 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3227648~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107145135000003, 2024-05-08T08:27:11.137 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033416003, 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108446 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.138 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994460+0000", 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994461+0000", 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2900000000000003e-07 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994463+0000", 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6570000000000001e-06 2024-05-08T08:27:11.139 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994470+0000", 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4489999999999996e-06 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994475+0000", 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1800000000000004e-06 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027699+0000", 2024-05-08T08:27:11.140 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033223821000000001 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027779+0000", 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0411e-05 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027876+0000", 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.6855999999999996e-05 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.141 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108447 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 3260416~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.107123090999998, 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033598876, 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108447 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.142 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994482+0000", 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994483+0000", 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8200000000000001e-07 2024-05-08T08:27:11.143 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994485+0000", 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994492+0000", 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9040000000000003e-06 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.994497+0000", 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3569999999999997e-06 2024-05-08T08:27:11.144 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027912+0000", 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.033415331999999999 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.027991+0000", 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8825999999999994e-05 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.028081+0000", 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9858000000000006e-05 2024-05-08T08:27:11.145 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108675 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6815744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.102127172000003, 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032085805000000002, 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.146 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108675 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999478+0000", 2024-05-08T08:27:11.147 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999480+0000", 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.793e-06 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999486+0000", 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6080000000000003e-06 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.148 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999490+0000", 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.464e-06 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031407+0000", 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031916959000000002 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031488+0000", 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0567000000000003e-05 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.149 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031564+0000", 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6114999999999999e-05 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108676 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6848512~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.102109673999998, 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032251646000000002, 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.150 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108676 2024-05-08T08:27:11.151 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.152 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999496+0000", 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9999999999999999e-07 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999498+0000", 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9199999999999998e-06 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999503+0000", 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1540000000000001e-06 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.153 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999509+0000", 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8270000000000003e-06 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031594+0000", 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032084797999999998 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031672+0000", 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8765999999999998e-05 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.154 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031747+0000", 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4881000000000004e-05 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108677 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6881280~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.102090918000002, 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032405850999999999, 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.155 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108677 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999514+0000", 2024-05-08T08:27:11.156 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999515+0000", 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.79e-07 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999516+0000", 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8279999999999999e-06 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.157 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999522+0000", 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2460000000000003e-06 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999526+0000", 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.673e-06 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031777+0000", 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032250681000000003 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.158 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031851+0000", 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3694000000000003e-05 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031920+0000", 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9449999999999994e-05 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108678 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6914048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:11.159 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.102073613000002, 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032573421999999998, 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108678 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.160 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999532+0000", 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0800000000000004e-07 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999538+0000", 2024-05-08T08:27:11.161 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2149999999999998e-06 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999545+0000", 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:19.999549+0000", 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5770000000000004e-06 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.162 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.031947+0000", 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032397903999999998 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032029+0000", 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1985999999999997e-05 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032105+0000", 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6032999999999995e-05 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.163 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108703 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6832128~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.100746268999998, 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.03217747, 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108703 2024-05-08T08:27:11.164 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.165 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000859+0000", 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9900000000000002e-07 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000861+0000", 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8199999999999999e-06 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000866+0000", 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.0880000000000002e-06 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.166 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.000871+0000", 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3510000000000004e-06 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032860+0000", 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031989640999999999 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.032962+0000", 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00010197200000000001 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.167 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033036+0000", 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4299000000000001e-05 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108704 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6864896~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.100397164999997, 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032008086999999998, 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.168 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108704 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001208+0000", 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.169 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001209+0000", 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5199999999999996e-07 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001213+0000", 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8770000000000003e-06 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001224+0000", 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1342000000000001e-05 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.170 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001231+0000", 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.306e-06 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033067+0000", 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031835544 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033149+0000", 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1719999999999997e-05 2024-05-08T08:27:11.171 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033216+0000", 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7645999999999996e-05 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108705 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6897664~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.100361808999999, 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032150629999999999, 2024-05-08T08:27:11.172 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108705 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.173 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001243+0000", 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001244+0000", 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6600000000000002e-07 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001247+0000", 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.83e-06 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.174 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001254+0000", 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7190000000000001e-06 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001260+0000", 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5099999999999998e-06 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033245+0000", 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.031984929000000002 2024-05-08T08:27:11.175 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033322+0000", 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7435999999999995e-05 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033394+0000", 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1740000000000001e-05 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.176 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:108706 3.4 3:22622b1e:::rbd_data.11918d3cc87b.0000000000000002:25 [read 6930432~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e84)", 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 51.100334115999999, 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032385262999999997, 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 108706 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:27:11.177 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001271+0000", 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001272+0000", 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6300000000000005e-07 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001274+0000", 2024-05-08T08:27:11.178 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7130000000000001e-06 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001281+0000", 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0840000000000003e-06 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.001287+0000", 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2059999999999998e-06 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.179 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033490+0000", 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.032203455999999998 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033573+0000", 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2860000000000003e-05 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:20.033656+0000", 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3380999999999993e-05 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.180 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.181 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.181 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:11.182 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T08:27:11.183 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:11.188+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:11.282 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:11.289+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:11.333 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:11.333 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:11.333 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:11.342 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T08:27:11.383 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:11.390+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:11.484 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:11.492+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:11.491 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:11.491 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:11.491 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:11.491 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:11.491 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:11.500 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T08:27:11.585 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:11.592+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:11.650 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69714 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 2932736~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836696063999995, 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046246284999999998, 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.651 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69714 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820800+0000", 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.652 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820801+0000", 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.72e-07 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820803+0000", 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2170000000000001e-06 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820809+0000", 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7410000000000003e-06 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.653 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820815+0000", 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6200000000000004e-06 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866947+0000", 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046131923999999998 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.866993+0000", 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6359000000000001e-05 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867046+0000", 2024-05-08T08:27:11.654 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3052e-05 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69715 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3162112~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836673908999998, 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046369965999999999, 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.655 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69715 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820822+0000", 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.656 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820823+0000", 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820825+0000", 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.655e-06 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820832+0000", 2024-05-08T08:27:11.657 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0849999999999999e-06 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820838+0000", 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9329999999999998e-06 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867075+0000", 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046236823000000003 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.658 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867133+0000", 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.7639000000000002e-05 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867192+0000", 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9475999999999998e-05 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69716 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3391488~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:11.659 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836650409000001, 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046544169000000003, 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69716 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.660 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820846+0000", 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5400000000000002e-07 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820848+0000", 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1749999999999999e-06 2024-05-08T08:27:11.661 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820855+0000", 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9140000000000002e-06 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820861+0000", 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5400000000000003e-06 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867234+0000", 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046372732 2024-05-08T08:27:11.662 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867299+0000", 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5485000000000001e-05 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867390+0000", 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0969000000000003e-05 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.663 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69717 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3620864~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836628414000003, 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046712535999999999, 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69717 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.664 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820868+0000", 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.08e-07 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.665 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820870+0000", 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1320000000000001e-06 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820879+0000", 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6840000000000002e-06 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820892+0000", 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3356000000000001e-05 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.666 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867422+0000", 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046529981999999998 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867485+0000", 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2614999999999996e-05 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867580+0000", 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.5359000000000006e-05 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.667 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69718 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 3850240~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836595552999995, 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046863712000000002, 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.668 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69718 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.669 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820901+0000", 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9599999999999999e-07 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820904+0000", 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6000000000000001e-06 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820911+0000", 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5320000000000001e-06 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.670 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820917+0000", 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0619999999999997e-06 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867609+0000", 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046691734999999998 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.671 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867681+0000", 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1931e-05 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867764+0000", 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3356000000000006e-05 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.672 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69719 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4079616~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836571841999998, 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047037906999999997, 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69719 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.673 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820924+0000", 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820925+0000", 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8599999999999999e-07 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.674 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820927+0000", 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2060000000000001e-06 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820934+0000", 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.764e-06 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820939+0000", 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.553e-06 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.675 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867796+0000", 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.046856496999999997 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867868+0000", 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2163999999999996e-05 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867962+0000", 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4337000000000002e-05 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.676 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69720 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4308992~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836549696000006, 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047192222999999998, 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69720 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.677 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820947+0000", 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.06e-07 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.678 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820949+0000", 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3760000000000002e-06 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820956+0000", 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9319999999999999e-06 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820963+0000", 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.827e-06 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.679 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.867988+0000", 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047025083000000002 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868053+0000", 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4871999999999998e-05 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868139+0000", 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5626999999999999e-05 2024-05-08T08:27:11.680 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69721 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4538368~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836525295000001, 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047345685999999998, 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.681 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69721 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.682 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820971+0000", 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7099999999999997e-07 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820974+0000", 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3420000000000002e-06 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820982+0000", 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8370000000000003e-06 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.683 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820988+0000", 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9399999999999999e-06 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868167+0000", 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047179062000000001 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868232+0000", 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4889999999999997e-05 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.684 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868317+0000", 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4244000000000002e-05 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69722 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4767744~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836500614000002, 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047510657999999997, 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.685 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69722 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:11.686 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.820996+0000", 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6100000000000001e-07 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821004+0000", 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.61e-06 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.687 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821011+0000", 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6510000000000001e-06 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821019+0000", 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5569999999999998e-06 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868347+0000", 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047327859 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.688 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868419+0000", 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2725999999999995e-05 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868506+0000", 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6793999999999996e-05 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69723 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 4997120~16384] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:27:11.689 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836469097000005, 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047669162000000001, 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69723 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:27:11.690 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821027+0000", 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821028+0000", 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1399999999999997e-07 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821030+0000", 2024-05-08T08:27:11.691 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4949999999999998e-06 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821037+0000", 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.075e-06 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821043+0000", 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8499999999999999e-06 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.692 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868536+0000", 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047493103000000002 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868611+0000", 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4647999999999994e-05 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868696+0000", 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5476999999999995e-05 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.693 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69724 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5226496~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836445818000001, 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819591000000002, 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69724 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.694 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821050+0000", 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821051+0000", 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.41e-07 2024-05-08T08:27:11.695 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821053+0000", 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1349999999999999e-06 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821060+0000", 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.097e-06 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821066+0000", 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8089999999999998e-06 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.696 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868722+0000", 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047656233999999999 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868795+0000", 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2621999999999997e-05 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868870+0000", 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5352999999999995e-05 2024-05-08T08:27:11.697 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69725 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5455872~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836423526000004, 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047973017999999999, 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.698 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69725 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.699 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821073+0000", 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3300000000000003e-07 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821075+0000", 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.356e-06 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821082+0000", 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7680000000000003e-06 2024-05-08T08:27:11.700 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821088+0000", 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4310000000000002e-06 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868907+0000", 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047819382000000001 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.868979+0000", 2024-05-08T08:27:11.701 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1922999999999996e-05 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869046+0000", 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6724999999999996e-05 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69726 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5685248~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836384103, 2024-05-08T08:27:11.702 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048091651999999999, 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69726 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.703 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821112+0000", 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821113+0000", 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.37e-07 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821115+0000", 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7889999999999999e-06 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.704 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821124+0000", 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.191e-06 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821130+0000", 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1909999999999997e-06 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869065+0000", 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.047935187999999997 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.705 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869129+0000", 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4356999999999994e-05 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869204+0000", 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4399000000000004e-05 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.706 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69727 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 5914624~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836357996000004, 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048288931, 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69727 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.707 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821138+0000", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821139+0000", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.39e-07 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821141+0000", 2024-05-08T08:27:11.708 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.43e-06 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821148+0000", 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0129999999999996e-06 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821153+0000", 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3739999999999997e-06 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.709 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869256+0000", 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048102180000000001 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869343+0000", 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7828e-05 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869427+0000", 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.3666999999999997e-05 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.710 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69728 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6144000~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836335547999994, 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048487729, 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69728 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.711 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821161+0000", 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5499999999999999e-07 2024-05-08T08:27:11.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821164+0000", 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7930000000000002e-06 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821171+0000", 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9399999999999996e-06 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821176+0000", 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.575e-06 2024-05-08T08:27:11.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869489+0000", 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048312213999999999 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869566+0000", 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7657999999999994e-05 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869648+0000", 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2194000000000006e-05 2024-05-08T08:27:11.714 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69729 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6373376~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836312625999994, 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048680186, 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.715 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69729 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821184+0000", 2024-05-08T08:27:11.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7e-07 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821186+0000", 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1500000000000002e-06 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821192+0000", 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2720000000000003e-06 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821197+0000", 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9880000000000004e-06 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869699+0000", 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048501191999999999 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869782+0000", 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2973000000000007e-05 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869864+0000", 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2241000000000001e-05 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69730 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6602752~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836292427999993, 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048837093999999998, 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.719 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69730 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821204+0000", 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0200000000000003e-07 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821206+0000", 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0509999999999999e-06 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821212+0000", 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1240000000000002e-06 2024-05-08T08:27:11.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821218+0000", 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.254e-06 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869928+0000", 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048710003000000002 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.869995+0000", 2024-05-08T08:27:11.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7125000000000005e-05 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870041+0000", 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.6134999999999998e-05 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69731 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 6832128~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836271355999997, 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048956353000000001, 2024-05-08T08:27:11.723 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69731 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821225+0000", 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6800000000000001e-07 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821228+0000", 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4880000000000001e-06 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821234+0000", 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.2299999999999996e-06 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821239+0000", 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.485e-06 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870066+0000", 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048826621000000001 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870133+0000", 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6607999999999997e-05 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870181+0000", 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8553000000000001e-05 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69732 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7061504~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:11.727 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836232467000002, 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049133148000000001, 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69732 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:11.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821264+0000", 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.82e-07 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.729 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821267+0000", 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7539999999999998e-06 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821286+0000", 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9122000000000001e-05 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821292+0000", 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3670000000000002e-06 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870231+0000", 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.048938161000000001 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870316+0000", 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5143999999999997e-05 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870397+0000", 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1118000000000005e-05 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:69733 3.1b 3:de987f22:::rbd_data.1191d80f8e36.0000000000000000:1e [read 7290880~0] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e70)", 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 69.836195141999994, 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049266982000000001, 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 69733 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:11.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821301+0000", 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821302+0000", 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7100000000000002e-07 2024-05-08T08:27:11.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821304+0000", 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.5409999999999999e-06 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821311+0000", 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1729999999999998e-06 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:11.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.821317+0000", 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2660000000000001e-06 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870426+0000", 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.049109685 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870497+0000", 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0860999999999998e-05 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:01.870568+0000", 2024-05-08T08:27:11.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0984999999999999e-05 2024-05-08T08:27:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:11.736 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:11.736 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:11.737 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T08:27:11.738 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:11.744+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:11.838 INFO:tasks.ceph.osd.5.smithi149.stderr:2024-05-08T08:27:11.845+0000 7f3c25e92640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32421) UID: 0 2024-05-08T08:27:11.887 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:11.887 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:11.887 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:11.888 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:11.896 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T08:27:11.940 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:11.948+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:12.005 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:12.005 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:12.005 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:12.014 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T08:27:12.039 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:12.047+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:12.104 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:27:12.148 INFO:tasks.ceph.osd.1.smithi031.stderr:2024-05-08T08:27:12.156+0000 7f005ce14640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 44733) UID: 0 2024-05-08T08:27:12.187 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:12.187 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2381 3.0 e24/18 3:06849f09:::rbd_data.119116cc7dd0.000000000000038e:head v 24'46, mlcod=24'42)", 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 120.468583462, 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043450229999999999, 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.188 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725500+0000", 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725501+0000", 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8700000000000004e-07 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725508+0000", 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6480000000000003e-06 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.189 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725524+0000", 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5908000000000001e-05 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725533+0000", 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9220000000000003e-06 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725601+0000", 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7644000000000005e-05 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.190 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.725625+0000", 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4023999999999999e-05 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729780+0000", 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041556689999999999 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.729845+0000", 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4320999999999996e-05 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.191 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:2640 3.4 e24/18 3:24f8193d:::rbd_data.1191a2472683.0000000000000001:head v 24'48, mlcod=24'46)", 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 120.198889925, 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0044716160000000003, 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.192 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995193+0000", 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.995195+0000", 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.407e-06 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996157+0000", 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00096237999999999998 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.193 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996177+0000", 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0245999999999999e-05 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996186+0000", 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.8140000000000007e-06 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996262+0000", 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5451999999999995e-05 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.194 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.996290+0000", 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8296000000000001e-05 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999619+0000", 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0033295360000000001 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:11.999665+0000", 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5485000000000002e-05 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.195 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:14469 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 107.054650463, 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039156959999999998, 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 14469 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.196 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139433+0000", 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139435+0000", 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0480000000000001e-06 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139443+0000", 2024-05-08T08:27:12.197 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.653e-06 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139464+0000", 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0971e-05 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139481+0000", 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6166000000000001e-05 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139556+0000", 2024-05-08T08:27:12.198 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5574000000000005e-05 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.139564+0000", 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6029999999999999e-06 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141319+0000", 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0017551190000000001 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.141394+0000", 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4608000000000001e-05 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.199 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143215+0000", 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0018214329999999999 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143271+0000", 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5835999999999999e-05 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:25.143348+0000", 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7684999999999999e-05 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.200 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20567 3.0 3:07e861f4:::rbd_data.119174045df.000000000000000c:head [sparse-read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.93100404099999, 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045360009999999996, 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20567 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.201 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263079+0000", 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263081+0000", 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.9759999999999998e-06 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.202 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263090+0000", 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5620000000000004e-06 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263111+0000", 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1767999999999999e-05 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263123+0000", 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1061e-05 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.203 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263236+0000", 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000113239 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263244+0000", 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8890000000000002e-06 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267404+0000", 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041603639999999997 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.204 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267480+0000", 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5696000000000002e-05 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267615+0000", 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000135446 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:20581 3.0 3:03d5d585:::rbd_data.119174045df.000000000000001a:head [sparse-read 0~1048576] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:27:12.205 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.930587164, 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0042389749999999999, 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 20581 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:27:12.206 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263496+0000", 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263497+0000", 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2799999999999995e-07 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263502+0000", 2024-05-08T08:27:12.207 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2120000000000002e-06 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263515+0000", 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2778e-05 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263522+0000", 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.2259999999999999e-06 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263609+0000", 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7368999999999996e-05 2024-05-08T08:27:12.208 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting_for_map not empty", 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.263615+0000", 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0920000000000003e-06 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267645+0000", 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0040293789999999996 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.267735+0000", 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0191000000000004e-05 2024-05-08T08:27:12.209 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:21118 3.0 3:07b885d6:::rbd_data.1191948f0501.0000000000000038:e [read 487424~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.71646576800001, 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036597800000000001, 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.210 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 21118 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477617+0000", 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.211 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477619+0000", 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5859999999999999e-06 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477621+0000", 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2970000000000002e-06 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477626+0000", 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8119999999999998e-06 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.212 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.477631+0000", 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.3980000000000001e-06 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480784+0000", 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.003153028 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.480857+0000", 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3243000000000003e-05 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.213 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.481277+0000", 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00042041600000000001 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:22050 3.0 3:0059c759:::rbd_data.1191daf6073a.0000000000000005:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e40)", 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 102.424251938, 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015499234000000001, 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.214 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 22050 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769831+0000", 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.215 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769833+0000", 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.962e-06 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769844+0000", 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.0767e-05 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769864+0000", 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0024e-05 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.216 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769873+0000", 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.3980000000000005e-06 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.769949+0000", 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5427999999999997e-05 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.770020+0000", 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0832000000000002e-05 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:12.217 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785236+0000", 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.015216018 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785277+0000", 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1757e-05 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:29.785331+0000", 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3047999999999998e-05 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.218 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23285 3.0 3:0528ea9a:::rbd_data.1191ea0f1b1f.0000000000000005:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 16777216 write_size 16777216,truncate 11534336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 101.697049902, 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.011155824999999999, 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23285 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.219 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497033+0000", 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.497035+0000", 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.858e-06 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.220 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499352+0000", 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0023165400000000002 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499378+0000", 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6108000000000001e-05 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499392+0000", 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4525000000000001e-05 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.221 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499458+0000", 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5982000000000006e-05 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.499577+0000", 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011845600000000001 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508073+0000", 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0084966880000000005 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508125+0000", 2024-05-08T08:27:12.222 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1827999999999999e-05 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.508189+0000", 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3839999999999999e-05 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:23434 3.4 3:247c74cb:::rbd_header.1191ea0f1b1f:head [call rbd.set_size in=11b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e41)", 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 101.478069537, 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0046252690000000004, 2024-05-08T08:27:12.223 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 23434 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.224 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716014+0000", 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4499999999999997e-07 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716018+0000", 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.242e-06 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716025+0000", 2024-05-08T08:27:12.225 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9460000000000001e-06 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716029+0000", 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9859999999999996e-06 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716205+0000", 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00017651599999999999 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.716260+0000", 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4917999999999998e-05 2024-05-08T08:27:12.226 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720561+0000", 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011860000000002 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720592+0000", 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0617999999999997e-05 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:30.720639+0000", 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7212000000000001e-05 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.227 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:27266 3.9 e48/18 3:966749f1:::rbd_data.119137f21733.000000000000035f:head v 48'193, mlcod=48'189)", 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 93.989844172000005, 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041147129999999999, 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.228 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204239+0000", 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204240+0000", 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8599999999999998e-07 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.229 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204249+0000", 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0799999999999995e-06 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204258+0000", 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.0650000000000005e-06 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204263+0000", 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3759999999999999e-06 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.230 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204322+0000", 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9002999999999997e-05 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.204347+0000", 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4666999999999999e-05 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208304+0000", 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039574800000000002 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.231 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:38.208354+0000", 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9356e-05 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:31363 3.0 3:02c41fe9:::rbd_data.119198308290.0000000000000103:head [set-alloc-hint object_size 524288 write_size 524288,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e53)", 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 89.069580017000007, 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0067675549999999998, 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.232 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 31363 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124503+0000", 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.233 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124505+0000", 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4729999999999999e-06 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124513+0000", 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4840000000000004e-06 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124530+0000", 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7161999999999999e-05 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.234 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124540+0000", 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.7729999999999994e-06 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124609+0000", 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9185999999999998e-05 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.124777+0000", 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000167787 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131184+0000", 2024-05-08T08:27:12.235 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.006407037 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131221+0000", 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7270000000000001e-05 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:43.131271+0000", 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9382999999999998e-05 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.236 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:33617 3.0 e58/57 3:06fc9e01:::rbd_data.11915c46bf0c.0000000000000002:head v 58'186, mlcod=0'0)", 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.921573882999994, 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.010109705, 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.237 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272509+0000", 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.272511+0000", 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.8509999999999999e-06 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274022+0000", 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0015104109999999999 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.238 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274048+0000", 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6653e-05 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274061+0000", 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.291e-05 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274179+0000", 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00011757 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.239 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.274209+0000", 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0471999999999999e-05 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282566+0000", 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0083569290000000008 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.282619+0000", 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2908999999999998e-05 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.240 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:34931 3.9 e58/18 3:92e442d9:::rbd_data.11915c46bf0c.0000000000000014:head v 58'214, mlcod=56'212)", 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 83.830712989000006, 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0050470940000000002, 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.241 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363370+0000", 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.363371+0000", 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9799999999999999e-07 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365804+0000", 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024332239999999999 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.242 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365824+0000", 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0375000000000001e-05 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365833+0000", 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5609999999999999e-06 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365903+0000", 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9852999999999997e-05 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.243 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.365927+0000", 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4230999999999999e-05 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368372+0000", 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0024446260000000001 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:48.368417+0000", 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5825999999999998e-05 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.244 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:89326 3.0 e76/57 3:063ad70a:::rbd_data.1191eeab3f85.000000000000013a:head v 76'296, mlcod=75'294)", 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 61.449582745999997, 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041412080000000004, 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.245 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744500+0000", 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744501+0000", 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4600000000000004e-07 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744507+0000", 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.9710000000000001e-06 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.246 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744520+0000", 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2644e-05 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744527+0000", 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5889999999999997e-06 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744589+0000", 2024-05-08T08:27:12.247 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1461e-05 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.744620+0000", 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.0964999999999999e-05 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748601+0000", 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0039815859999999996 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:10.748642+0000", 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0346000000000003e-05 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.248 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113547 3.0 e99/57 3:012c0039:::rbd_data.11918629634e.0000000000000279:head v 99'328, mlcod=99'325)", 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.078402578999999, 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0043011179999999996, 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:12.249 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115681+0000", 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115687+0000", 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.293e-06 2024-05-08T08:27:12.250 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115699+0000", 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-05 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115705+0000", 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.3860000000000004e-06 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115753+0000", 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7938000000000001e-05 2024-05-08T08:27:12.251 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.115770+0000", 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.6809000000000001e-05 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119940+0000", 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0041703799999999996 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.119982+0000", 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.1530999999999999e-05 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.252 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(client.4497.0:113549 3.9 e99/18 3:90ba2507:::rbd_data.11918629634e.000000000000027b:head v 99'310, mlcod=99'308)", 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 37.076102999, 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0047200790000000003, 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.253 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117980+0000", 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117981+0000", 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7100000000000001e-07 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.117988+0000", 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0709999999999997e-06 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.254 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118003+0000", 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.4701999999999999e-05 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118010+0000", 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5619999999999998e-06 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118059+0000", 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8198999999999997e-05 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.255 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.118074+0000", 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5262000000000001e-05 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122655+0000", 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0045809400000000004 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:35.122700+0000", 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.5571999999999997e-05 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.256 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:114608 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 29.079344959, 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95824956900000002, 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 114608 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.257 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114738+0000", 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114740+0000", 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5069999999999999e-06 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.258 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114747+0000", 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9380000000000003e-06 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114762+0000", 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5464999999999998e-05 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114770+0000", 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8599999999999993e-06 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114842+0000", 2024-05-08T08:27:12.259 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1748999999999994e-05 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:43.114856+0000", 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3976e-05 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071293+0000", 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.95643727300000003 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071421+0000", 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000127718 2024-05-08T08:27:12.260 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.071573+0000", 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015250100000000001 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072356+0000", 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000783141 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072850+0000", 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00049327699999999997 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.261 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072933+0000", 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2993999999999999e-05 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:44.072988+0000", 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.5170000000000002e-05 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:543 3.0 e109/57 3:04449ea6:::rbd_data.1191f60358f6.000000000000000c:14 v 109'338, mlcod=102'335)", 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.097770865000001, 2024-05-08T08:27:12.262 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0037001569999999999, 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096312+0000", 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.263 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096313+0000", 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.8100000000000003e-07 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096319+0000", 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.251e-06 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096332+0000", 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3107e-05 2024-05-08T08:27:12.264 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096341+0000", 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.619e-06 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096414+0000", 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.3201000000000006e-05 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.096437+0000", 2024-05-08T08:27:12.265 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2891000000000002e-05 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.099956+0000", 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0035193490000000002 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.100013+0000", 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6258000000000002e-05 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.266 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_repop(osd.1.0:544 3.0 e109/57 3:02813c57:::rbd_data.1191f60358f6.0000000000000058:14 v 109'340, mlcod=102'335)", 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 27.097000352999999, 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0057648059999999999, 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.267 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097083+0000", 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097084+0000", 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5099999999999999e-07 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097092+0000", 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5189999999999992e-06 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.268 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097105+0000", 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2685e-05 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097112+0000", 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0049999999999998e-06 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097176+0000", 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4166999999999997e-05 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.269 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.097203+0000", 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6630000000000001e-05 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102791+0000", 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0055883590000000002 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:45.102848+0000", 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.6789999999999997e-05 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.270 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:115393 3.3 3:c4c92e5a:::rbd_trash:head [call rbd.trash_add in=12b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118)", 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 18.15688471, 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0036588620000000001, 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 115393 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.271 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037199+0000", 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037200+0000", 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.86e-06 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037209+0000", 2024-05-08T08:27:12.272 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.7090000000000008e-06 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037229+0000", 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0049000000000001e-05 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037238+0000", 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.6640000000000004e-06 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037305+0000", 2024-05-08T08:27:12.273 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7300999999999998e-05 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op must wait for map", 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.037311+0000", 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.1029999999999998e-06 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039469+0000", 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0021582839999999999 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.274 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039564+0000", 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.4547000000000001e-05 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for subops from 1", 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.039753+0000", 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00018868800000000001 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "op_commit", 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040757+0000", 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.0010045869999999999 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.275 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "sub_op_commit_rec", 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040784+0000", 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6840999999999999e-05 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "commit_sent", 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040824+0000", 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.9956000000000002e-05 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:54.040857+0000", 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3272999999999998e-05 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.276 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.277 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.277 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:12.278 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T08:27:12.278 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:12.285+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:12.379 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:12.387+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:12.429 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:12.429 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:12.429 INFO:teuthology.orchestra.run.smithi031.stdout: "num_ops": 0 2024-05-08T08:27:12.429 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:12.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T08:27:12.480 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:12.488+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:12.542 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T08:27:12.581 INFO:tasks.ceph.osd.7.smithi149.stderr:2024-05-08T08:27:12.589+0000 7f4c587b5640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32428) UID: 0 2024-05-08T08:27:12.587 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [], 2024-05-08T08:27:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "complaint_time": 30, 2024-05-08T08:27:12.587 INFO:teuthology.orchestra.run.smithi031.stdout: "num_blocked_ops": 0 2024-05-08T08:27:12.587 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:12.597 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T08:27:12.670 INFO:teuthology.orchestra.run.smithi031.stdout:osd.0: {} 2024-05-08T08:27:12.671 INFO:teuthology.orchestra.run.smithi031.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T08:27:12.682 INFO:tasks.ceph.osd.2.smithi031.stderr:2024-05-08T08:27:12.689+0000 7f84b6437640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32765) UID: 0 2024-05-08T08:27:12.689 INFO:teuthology.orchestra.run.smithi031.stdout:osd.1: {} 2024-05-08T08:27:12.689 INFO:teuthology.orchestra.run.smithi031.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T08:27:12.706 INFO:teuthology.orchestra.run.smithi031.stdout:osd.2: {} 2024-05-08T08:27:12.706 INFO:teuthology.orchestra.run.smithi031.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "size": 20, 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 600, 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "ops": [ 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54376 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 943104~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557409958999997, 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060952023000000001, 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.711 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54376 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.712 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160091+0000", 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.05e-07 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160093+0000", 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.217e-06 2024-05-08T08:27:12.713 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160096+0000", 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.909e-06 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160098+0000", 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.706e-06 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.714 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220908+0000", 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060809433000000003 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.220979+0000", 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1527999999999997e-05 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221043+0000", 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4024999999999998e-05 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.715 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54377 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 952320~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557399873999998, 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061107158000000002, 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54377 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.716 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160101+0000", 2024-05-08T08:27:12.717 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0599999999999999e-07 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160103+0000", 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2109999999999999e-06 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160106+0000", 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.92e-06 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160108+0000", 2024-05-08T08:27:12.718 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.429e-06 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221071+0000", 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060963133000000003 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221143+0000", 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.1602999999999999e-05 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221208+0000", 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5655999999999996e-05 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.719 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54378 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 961536~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557389395000001, 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.06127519, 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.720 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54378 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.721 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160112+0000", 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.03e-07 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160114+0000", 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.885e-06 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.722 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160118+0000", 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0709999999999996e-06 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160122+0000", 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.2910000000000001e-06 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221235+0000", 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061113171000000001 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221312+0000", 2024-05-08T08:27:12.723 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6925e-05 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221387+0000", 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4543999999999996e-05 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54379 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 970752~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.724 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557372469000001, 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061445306999999998, 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54379 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.725 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160129+0000", 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4099999999999999e-07 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160132+0000", 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4940000000000002e-06 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.726 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160138+0000", 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0569999999999998e-06 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160142+0000", 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.7010000000000004e-06 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221418+0000", 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061275266000000002 2024-05-08T08:27:12.727 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221496+0000", 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8529000000000007e-05 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221574+0000", 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7719000000000006e-05 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.728 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54380 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 979968~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557351549000003, 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061608804000000003, 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54380 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.729 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.730 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160150+0000", 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.8000000000000001e-07 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160165+0000", 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5226e-05 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160174+0000", 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.5159999999999994e-06 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.731 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160179+0000", 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.587e-06 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221605+0000", 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061425647999999999 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221681+0000", 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.5740000000000003e-05 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.732 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221758+0000", 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.7707000000000007e-05 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54381 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 989184~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557313234000006, 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061762360000000002, 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.733 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54381 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:12.734 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160188+0000", 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.9999999999999998e-07 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160191+0000", 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.977e-06 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.735 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160198+0000", 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8310000000000003e-06 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160204+0000", 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.8030000000000002e-06 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221788+0000", 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061583876000000003 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.736 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221866+0000", 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8129999999999999e-05 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221950+0000", 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4243e-05 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54382 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 998400~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.737 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557289487999995, 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061923884999999998, 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54382 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.738 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160212+0000", 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5100000000000001e-07 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.739 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160214+0000", 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4789999999999999e-06 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160222+0000", 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0729999999999999e-06 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160227+0000", 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.49e-06 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.740 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221983+0000", 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061755908999999998 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222065+0000", 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.2336999999999995e-05 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222136+0000", 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.0246000000000005e-05 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.741 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54383 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1007616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557266644999999, 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062080980000000001, 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.742 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54383 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160234+0000", 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.743 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160235+0000", 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5999999999999999e-07 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160237+0000", 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.3080000000000002e-06 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160244+0000", 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.5910000000000001e-06 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.744 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160249+0000", 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3000000000000001e-06 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222165+0000", 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061916082999999997 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222246+0000", 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.1091e-05 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.745 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222315+0000", 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9246999999999996e-05 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54384 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1016832~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557242255999995, 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062229598999999997, 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.746 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54384 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.747 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160259+0000", 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0900000000000002e-07 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160262+0000", 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2680000000000001e-06 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160269+0000", 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9789999999999996e-06 2024-05-08T08:27:12.748 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160274+0000", 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.1760000000000001e-06 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222344+0000", 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062070409 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222419+0000", 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.4720000000000003e-05 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.749 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222488+0000", 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.9638e-05 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54385 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1026048~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557220336, 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062430095999999997, 2024-05-08T08:27:12.750 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54385 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.751 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160281+0000", 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.7899999999999999e-07 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160283+0000", 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2919999999999998e-06 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.752 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160290+0000", 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7050000000000002e-06 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160295+0000", 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2279999999999998e-06 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222519+0000", 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062224069 2024-05-08T08:27:12.753 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222596+0000", 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.6536e-05 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222711+0000", 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000114887 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.754 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54386 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1035264~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557199101999998, 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062748867, 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54386 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.755 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.756 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160302+0000", 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.6600000000000002e-07 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160305+0000", 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2560000000000001e-06 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160311+0000", 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.7349999999999999e-06 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.757 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160317+0000", 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.446e-06 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222741+0000", 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062424089000000002 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222822+0000", 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0852000000000004e-05 2024-05-08T08:27:12.758 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223051+0000", 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00022912299999999999 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54387 3.7 3:e7e50c78:::rbd_data.11914ad0077c.0000000000000003:1a [read 1044480~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.557177658000001, 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062903629000000003, 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.759 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54387 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.760 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160323+0000", 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160324+0000", 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.5900000000000003e-07 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160326+0000", 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.1799999999999999e-06 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.761 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160333+0000", 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6440000000000003e-06 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.160339+0000", 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0580000000000003e-06 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223079+0000", 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062739959999999997 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.762 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223159+0000", 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0031000000000007e-05 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223227+0000", 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8397000000000003e-05 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54495 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 990208~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:12.763 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.556193354000001, 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061304558000000002, 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54495 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.764 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.765 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161308+0000", 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.84e-07 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161311+0000", 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6359999999999998e-06 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161317+0000", 2024-05-08T08:27:12.766 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.4829999999999996e-06 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161323+0000", 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.2390000000000002e-06 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.221977+0000", 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060654155000000001 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.767 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222111+0000", 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013450100000000001 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222612+0000", 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00050115999999999995 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.768 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54496 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 999424~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.556172153000006, 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061211408000000002, 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54496 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.769 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.770 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161329+0000", 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6100000000000002e-07 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161331+0000", 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.7179999999999999e-06 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.771 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161334+0000", 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 3.3119999999999998e-06 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161337+0000", 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.7389999999999999e-06 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222325+0000", 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.060987940999999997 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.772 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222456+0000", 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000131309 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222540+0000", 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.4128000000000005e-05 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54497 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1008640~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.556160683000002, 2024-05-08T08:27:12.773 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061463260999999998, 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54497 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.774 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161340+0000", 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161341+0000", 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.775 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161342+0000", 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.2130000000000001e-06 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161345+0000", 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8009999999999999e-06 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161347+0000", 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.526e-06 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222569+0000", 2024-05-08T08:27:12.776 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061221339999999999 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222723+0000", 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00015452900000000001 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222804+0000", 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.0644999999999998e-05 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.777 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54498 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1017856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.556150923000004, 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061703758999999997, 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54498 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.778 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161350+0000", 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.0699999999999999e-07 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.779 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161352+0000", 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1400000000000001e-06 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161354+0000", 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.74e-06 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161357+0000", 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4820000000000001e-06 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.780 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222835+0000", 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061478194 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.222925+0000", 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9801000000000003e-05 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223054+0000", 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.000129195 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.781 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54499 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1027072~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.556141134000001, 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061872779000000003, 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.782 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54499 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.783 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161360+0000", 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2000000000000001e-07 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161362+0000", 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.3370000000000001e-06 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161365+0000", 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.954e-06 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.784 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161367+0000", 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.9170000000000002e-06 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223086+0000", 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061718644000000003 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223166+0000", 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.9938999999999995e-05 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.785 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223233+0000", 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.6767999999999995e-05 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54500 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1036288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.556130609999997, 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062061231000000001, 2024-05-08T08:27:12.786 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54500 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.787 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161371+0000", 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.03e-07 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161372+0000", 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.212e-06 2024-05-08T08:27:12.788 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161375+0000", 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.8739999999999999e-06 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161377+0000", 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.4480000000000001e-06 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223264+0000", 2024-05-08T08:27:12.789 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.061886676000000002 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223343+0000", 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.8615999999999994e-05 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223432+0000", 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 8.9202000000000004e-05 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.790 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:54501 3.18 3:1f68f935:::rbd_data.11914ad0077c.0000000000000007:1a [read 1045504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 80.556120930000006, 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062295370000000003, 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "started", 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.791 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 54501 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.792 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161380+0000", 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.2499999999999999e-07 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161382+0000", 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.1179999999999999e-06 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161386+0000", 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 4.0400000000000003e-06 2024-05-08T08:27:12.793 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.161388+0000", 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 2.6869999999999999e-06 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223486+0000", 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.062098027 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "started", 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223541+0000", 2024-05-08T08:27:12.794 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.4459999999999997e-05 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:25:52.223676+0000", 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.00013481300000000001 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "description": "osd_op(client.4497.0:102187 3.a 3.fa1caca (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e80)", 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "initiated_at": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "age": 57.332859781000003, 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92937441300000001, 2024-05-08T08:27:12.795 INFO:teuthology.orchestra.run.smithi031.stdout: "continuous": false, 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "type_data": { 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "flag_point": "reached pg", 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_info": { 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client": "client.4497", 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "client_addr": "172.21.15.31:0/3546803534", 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "tid": 102187 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "events": [ 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "initiated", 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "header_read", 2024-05-08T08:27:12.796 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384641+0000", 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "throttled", 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384643+0000", 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.595e-06 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "all_read", 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384650+0000", 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.8360000000000003e-06 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "dispatched", 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384665+0000", 2024-05-08T08:27:12.797 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 1.5645999999999999e-05 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "queued_for_pg", 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384673+0000", 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 7.678e-06 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384726+0000", 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 5.3012e-05 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "waiting for peered", 2024-05-08T08:27:12.798 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:15.384732+0000", 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 6.0990000000000004e-06 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "reached_pg", 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314007+0000", 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 0.92927435400000002 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: }, 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: { 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "event": "done", 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "time": "2024-05-08T08:26:16.314016+0000", 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: "duration": 9.1929999999999999e-06 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.799 INFO:teuthology.orchestra.run.smithi031.stdout: } 2024-05-08T08:27:12.800 INFO:teuthology.orchestra.run.smithi031.stdout: ] 2024-05-08T08:27:12.800 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:12.801 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T08:27:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:osd.3: {} 2024-05-08T08:27:12.802 INFO:teuthology.orchestra.run.smithi031.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T08:27:12.802 INFO:teuthology.orchestra.run.smithi031.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-05-08T08:27:12.802 INFO:teuthology.orchestra.run.smithi031.stdout:osd.5: {} 2024-05-08T08:27:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T08:27:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:osd.6: {} 2024-05-08T08:27:12.803 INFO:teuthology.orchestra.run.smithi031.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T08:27:12.803 INFO:teuthology.orchestra.run.smithi031.stdout:osd.7: {} 2024-05-08T08:27:12.804 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T08:27:12.805 INFO:teuthology.orchestra.run.smithi031.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T08:27:12.805 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T08:27:12.805 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:12.808+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:12.902 INFO:tasks.ceph.osd.3.smithi031.stderr:2024-05-08T08:27:12.909+0000 7f9025d7f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32774) UID: 0 2024-05-08T08:27:12.922 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:12.922 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:12.931 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T08:27:13.003 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:13.010+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:13.037 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.037 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.047 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T08:27:13.100 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:27:13.104 INFO:tasks.ceph.osd.0.smithi031.stderr:2024-05-08T08:27:13.111+0000 7faa82571640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32759) UID: 0 2024-05-08T08:27:13.197 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.197 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.202 INFO:teuthology.orchestra.run.smithi031.stdout:All operations completed A-OK! 2024-05-08T08:27:13.205 INFO:tasks.ceph.osd.6.smithi149.stderr:2024-05-08T08:27:13.212+0000 7f8063132640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32425) UID: 0 2024-05-08T08:27:13.207 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T08:27:13.219 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:13.220 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-05-08T08:27:13.229 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-05-08T08:27:13.247 INFO:tasks.thrashosds:joining thrashosds 2024-05-08T08:27:13.352 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.352 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.361 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T08:27:13.467 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.467 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.476 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T08:27:13.582 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.582 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.591 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T08:27:13.697 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.697 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.706 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T08:27:13.810 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.810 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.819 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T08:27:13.925 INFO:teuthology.orchestra.run.smithi149.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:13.925 INFO:teuthology.orchestra.run.smithi149.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:13.934 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T08:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:14.059 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:14.068 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T08:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:14.180 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:14.189 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T08:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout:ERROR: (22) Invalid argument 2024-05-08T08:27:14.299 INFO:teuthology.orchestra.run.smithi031.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T08:27:15.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T08:27:16.104 INFO:teuthology.orchestra.run.smithi031.stderr:noscrub is unset 2024-05-08T08:27:16.122 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T08:27:16.184 INFO:tasks.thrashosds.thrasher:reviving osd 2024-05-08T08:27:16.184 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-05-08T08:27:16.184 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T08:27:16.184 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T08:27:16.187 INFO:tasks.ceph.osd.4:Started 2024-05-08T08:27:16.187 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:no valid command found; 10 closest matches: 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:0 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:1 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:2 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:abort 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:assert 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:config diff 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:config diff get 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:config get 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:config help [] 2024-05-08T08:27:16.296 INFO:teuthology.orchestra.run.smithi149.stderr:config set ... 2024-05-08T08:27:16.297 INFO:teuthology.orchestra.run.smithi149.stderr:admin_socket: invalid command 2024-05-08T08:27:16.299 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T08:27:16.299 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-05-08T08:27:16.785 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:27:16.791+0000 7f93705a7640 -1 Falling back to public interface 2024-05-08T08:27:17.106 INFO:teuthology.orchestra.run.smithi031.stderr:nodeep-scrub is unset 2024-05-08T08:27:20.283 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:27:20.289+0000 7f93705a7640 -1 osd.4 108 log_to_monitors true 2024-05-08T08:27:21.142 INFO:tasks.ceph.osd.4.smithi149.stderr:2024-05-08T08:27:21.149+0000 7f936435d640 -1 osd.4 108 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T08:27:21.300 DEBUG:teuthology.orchestra.run.smithi149:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T08:27:21.418 INFO:teuthology.orchestra.run.smithi149.stdout:{ 2024-05-08T08:27:21.418 INFO:teuthology.orchestra.run.smithi149.stdout: "ops": [], 2024-05-08T08:27:21.418 INFO:teuthology.orchestra.run.smithi149.stdout: "num_ops": 0 2024-05-08T08:27:21.419 INFO:teuthology.orchestra.run.smithi149.stdout:} 2024-05-08T08:27:21.427 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-05-08T08:27:21.559 INFO:teuthology.orchestra.run.smithi031.stdout:mon.a: {} 2024-05-08T08:27:21.559 INFO:teuthology.orchestra.run.smithi031.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:27:21.569 INFO:teuthology.orchestra.run.smithi031.stdout:mon.b: {} 2024-05-08T08:27:21.569 INFO:teuthology.orchestra.run.smithi031.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:27:21.578 INFO:teuthology.orchestra.run.smithi031.stdout:mon.c: {} 2024-05-08T08:27:21.578 INFO:teuthology.orchestra.run.smithi031.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T08:27:21.590 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-08T08:27:22.124 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.0 to 1 (10000) 2024-05-08T08:27:22.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-05-08T08:27:23.126 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-05-08T08:27:23.148 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T08:27:24.140 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T08:27:24.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-05-08T08:27:25.145 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-05-08T08:27:25.164 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-08T08:27:26.147 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.3 to 1 (10000) 2024-05-08T08:27:26.170 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-05-08T08:27:27.166 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-05-08T08:27:27.184 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-08T08:27:28.161 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.5 to 1 (10000) 2024-05-08T08:27:28.185 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-05-08T08:27:28.977 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-05-08T08:27:28.997 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-08T08:27:29.983 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.6 to 1 (10000) 2024-05-08T08:27:30.001 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-05-08T08:27:31.181 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-05-08T08:27:31.198 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-05-08T08:27:32.186 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.7 to 1 (10000) 2024-05-08T08:27:32.203 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-05-08T08:27:33.198 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-05-08T08:27:33.215 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T08:27:34.203 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T08:27:34.221 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-05-08T08:27:35.213 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-05-08T08:27:35.230 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T08:27:36.218 INFO:teuthology.orchestra.run.smithi031.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T08:27:36.235 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-05-08T08:27:37.224 INFO:teuthology.orchestra.run.smithi031.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-05-08T08:27:37.240 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-05-08T08:27:37.241 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-05-08T08:27:37.241 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-05-08T08:27:37.241 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-05-08T08:27:37.241 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T08:27:37.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:27:37.548 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:37.549 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":158,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:27:37.227345+0000","last_up_change":"2024-05-08T08:27:22.131410+0000","last_in_change":"2024-05-08T08:27:04.049694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T08:24:55.134472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T08:24:56.465515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T08:25:03.054989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6801","nonce":727789107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6803","nonce":727789107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6807","nonce":727789107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6805","nonce":727789107}]},"public_addr":"172.21.15.31:6801/727789107","cluster_addr":"172.21.15.31:6803/727789107","heartbeat_back_addr":"172.21.15.31:6807/727789107","heartbeat_front_addr":"172.21.15.31:6805/727789107","state":["exists","up"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":30,"up_from":54,"up_thru":148,"down_at":32,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6817","nonce":2489759853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6819","nonce":2489759853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6823","nonce":2489759853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6821","nonce":2489759853}]},"public_addr":"172.21.15.31:6817/2489759853","cluster_addr":"172.21.15.31:6819/2489759853","heartbeat_back_addr":"172.21.15.31:6823/2489759853","heartbeat_front_addr":"172.21.15.31:6821/2489759853","state":["exists","up"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6809","nonce":2736372115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6811","nonce":2736372115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6815","nonce":2736372115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6813","nonce":2736372115}]},"public_addr":"172.21.15.31:6809/2736372115","cluster_addr":"172.21.15.31:6811/2736372115","heartbeat_back_addr":"172.21.15.31:6815/2736372115","heartbeat_front_addr":"172.21.15.31:6813/2736372115","state":["exists","up"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6825","nonce":573482428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6827","nonce":573482428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6831","nonce":573482428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6829","nonce":573482428}]},"public_addr":"172.21.15.31:6825/573482428","cluster_addr":"172.21.15.31:6827/573482428","heartbeat_back_addr":"172.21.15.31:6831/573482428","heartbeat_front_addr":"172.21.15.31:6829/573482428","state":["exists","up"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":108,"up_from":143,"up_thru":146,"down_at":110,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6803","nonce":2548527301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6807","nonce":2548527301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6815","nonce":2548527301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6811","nonce":2548527301}]},"public_addr":"172.21.15.149:6803/2548527301","cluster_addr":"172.21.15.149:6807/2548527301","heartbeat_back_addr":"172.21.15.149:6815/2548527301","heartbeat_front_addr":"172.21.15.149:6811/2548527301","state":["exists","up"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6818","nonce":3709461555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6822","nonce":3709461555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6830","nonce":3709461555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6826","nonce":3709461555}]},"public_addr":"172.21.15.149:6818/3709461555","cluster_addr":"172.21.15.149:6822/3709461555","heartbeat_back_addr":"172.21.15.149:6830/3709461555","heartbeat_front_addr":"172.21.15.149:6826/3709461555","state":["exists","up"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6802","nonce":405561966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6806","nonce":405561966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6814","nonce":405561966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6810","nonce":405561966}]},"public_addr":"172.21.15.149:6802/405561966","cluster_addr":"172.21.15.149:6806/405561966","heartbeat_back_addr":"172.21.15.149:6814/405561966","heartbeat_front_addr":"172.21.15.149:6810/405561966","state":["exists","up"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6817","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6819","nonce":570236217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6821","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6823","nonce":570236217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6829","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6831","nonce":570236217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6825","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6827","nonce":570236217}]},"public_addr":"172.21.15.149:6819/570236217","cluster_addr":"172.21.15.149:6823/570236217","heartbeat_back_addr":"172.21.15.149:6831/570236217","heartbeat_front_addr":"172.21.15.149:6827/570236217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.572981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.626543+0000","dead_epoch":32},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.638878+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.599293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.565676+0000","dead_epoch":110},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.552124+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.531428+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.658391+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.3","osds":[2,1]},{"pgid":"3.a","osds":[3,4]},{"pgid":"3.15","osds":[1,7]}],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":4}]},{"pgid":"3.8","mappings":[{"from":3,"to":4}]},{"pgid":"3.17","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:27:37.561 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T08:27:37.562 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:37.860 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":158,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:27:37.227345+0000","last_up_change":"2024-05-08T08:27:22.131410+0000","last_in_change":"2024-05-08T08:27:04.049694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T08:24:55.134472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T08:24:56.465515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T08:25:03.054989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6801","nonce":727789107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6803","nonce":727789107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6807","nonce":727789107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6805","nonce":727789107}]},"public_addr":"172.21.15.31:6801/727789107","cluster_addr":"172.21.15.31:6803/727789107","heartbeat_back_addr":"172.21.15.31:6807/727789107","heartbeat_front_addr":"172.21.15.31:6805/727789107","state":["exists","up"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":30,"up_from":54,"up_thru":148,"down_at":32,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6817","nonce":2489759853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6819","nonce":2489759853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6823","nonce":2489759853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6821","nonce":2489759853}]},"public_addr":"172.21.15.31:6817/2489759853","cluster_addr":"172.21.15.31:6819/2489759853","heartbeat_back_addr":"172.21.15.31:6823/2489759853","heartbeat_front_addr":"172.21.15.31:6821/2489759853","state":["exists","up"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6809","nonce":2736372115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6811","nonce":2736372115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6815","nonce":2736372115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6813","nonce":2736372115}]},"public_addr":"172.21.15.31:6809/2736372115","cluster_addr":"172.21.15.31:6811/2736372115","heartbeat_back_addr":"172.21.15.31:6815/2736372115","heartbeat_front_addr":"172.21.15.31:6813/2736372115","state":["exists","up"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6825","nonce":573482428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6827","nonce":573482428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6831","nonce":573482428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6829","nonce":573482428}]},"public_addr":"172.21.15.31:6825/573482428","cluster_addr":"172.21.15.31:6827/573482428","heartbeat_back_addr":"172.21.15.31:6831/573482428","heartbeat_front_addr":"172.21.15.31:6829/573482428","state":["exists","up"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":108,"up_from":143,"up_thru":146,"down_at":110,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6803","nonce":2548527301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6807","nonce":2548527301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6815","nonce":2548527301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6811","nonce":2548527301}]},"public_addr":"172.21.15.149:6803/2548527301","cluster_addr":"172.21.15.149:6807/2548527301","heartbeat_back_addr":"172.21.15.149:6815/2548527301","heartbeat_front_addr":"172.21.15.149:6811/2548527301","state":["exists","up"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6818","nonce":3709461555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6822","nonce":3709461555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6830","nonce":3709461555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6826","nonce":3709461555}]},"public_addr":"172.21.15.149:6818/3709461555","cluster_addr":"172.21.15.149:6822/3709461555","heartbeat_back_addr":"172.21.15.149:6830/3709461555","heartbeat_front_addr":"172.21.15.149:6826/3709461555","state":["exists","up"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6802","nonce":405561966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6806","nonce":405561966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6814","nonce":405561966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6810","nonce":405561966}]},"public_addr":"172.21.15.149:6802/405561966","cluster_addr":"172.21.15.149:6806/405561966","heartbeat_back_addr":"172.21.15.149:6814/405561966","heartbeat_front_addr":"172.21.15.149:6810/405561966","state":["exists","up"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6817","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6819","nonce":570236217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6821","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6823","nonce":570236217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6829","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6831","nonce":570236217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6825","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6827","nonce":570236217}]},"public_addr":"172.21.15.149:6819/570236217","cluster_addr":"172.21.15.149:6823/570236217","heartbeat_back_addr":"172.21.15.149:6831/570236217","heartbeat_front_addr":"172.21.15.149:6827/570236217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.572981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.626543+0000","dead_epoch":32},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.638878+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.599293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.565676+0000","dead_epoch":110},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.552124+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.531428+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.658391+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.3","osds":[2,1]},{"pgid":"3.a","osds":[3,4]},{"pgid":"3.15","osds":[1,7]}],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":4}]},{"pgid":"3.8","mappings":[{"from":3,"to":4}]},{"pgid":"3.17","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:27:37.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T08:27:37.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T08:27:37.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T08:27:37.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T08:27:37.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T08:27:37.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T08:27:37.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T08:27:37.875 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T08:27:38.136 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:38.136 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:27:38.137 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:38.137 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T08:27:38.138 INFO:teuthology.orchestra.run.smithi031.stdout:231928234008 2024-05-08T08:27:38.138 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:27:38.139 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:38.139 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:27:38.140 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:38.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T08:27:38.141 INFO:teuthology.orchestra.run.smithi031.stdout:55834574885 2024-05-08T08:27:38.141 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:27:38.142 INFO:teuthology.orchestra.run.smithi031.stdout:614180323333 2024-05-08T08:27:38.142 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:27:38.143 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:38.144 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:27:38.746 INFO:teuthology.orchestra.run.smithi031.stdout:55834574883 2024-05-08T08:27:38.767 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574883 for osd.2 2024-05-08T08:27:38.789 INFO:teuthology.orchestra.run.smithi031.stdout:55834574883 2024-05-08T08:27:38.803 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574883 for osd.6 2024-05-08T08:27:38.808 INFO:teuthology.orchestra.run.smithi031.stdout:55834574883 2024-05-08T08:27:38.813 INFO:teuthology.orchestra.run.smithi031.stdout:55834574883 2024-05-08T08:27:38.815 INFO:teuthology.orchestra.run.smithi031.stdout:231928234007 2024-05-08T08:27:38.827 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:38.828 INFO:teuthology.orchestra.run.smithi031.stdout:614180323332 2024-05-08T08:27:38.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574883 for osd.7 2024-05-08T08:27:38.832 INFO:teuthology.orchestra.run.smithi031.stdout:55834574883 2024-05-08T08:27:38.833 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574883 for osd.5 2024-05-08T08:27:38.834 INFO:tasks.ceph.ceph_manager.ceph:need seq 231928234008 got 231928234007 for osd.1 2024-05-08T08:27:38.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 614180323333 got 614180323332 for osd.4 2024-05-08T08:27:38.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574885 got 55834574884 for osd.3 2024-05-08T08:27:38.847 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574883 for osd.0 2024-05-08T08:27:39.768 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T08:27:39.804 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T08:27:39.829 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T08:27:39.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T08:27:39.834 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T08:27:39.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T08:27:39.846 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T08:27:39.847 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T08:27:40.346 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:40.366 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574884 for osd.2 2024-05-08T08:27:40.366 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.425 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:40.441 INFO:teuthology.orchestra.run.smithi031.stdout:614180323333 2024-05-08T08:27:40.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574884 for osd.6 2024-05-08T08:27:40.446 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.453 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:40.457 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:40.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 614180323333 got 614180323333 for osd.4 2024-05-08T08:27:40.461 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574884 for osd.0 2024-05-08T08:27:40.468 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574884 for osd.7 2024-05-08T08:27:40.476 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.486 INFO:teuthology.orchestra.run.smithi031.stdout:55834574885 2024-05-08T08:27:40.493 INFO:teuthology.orchestra.run.smithi031.stdout:231928234009 2024-05-08T08:27:40.498 INFO:teuthology.orchestra.run.smithi031.stdout:55834574884 2024-05-08T08:27:40.500 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574885 got 55834574885 for osd.3 2024-05-08T08:27:40.500 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.509 INFO:tasks.ceph.ceph_manager.ceph:need seq 231928234008 got 231928234009 for osd.1 2024-05-08T08:27:40.509 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574884 got 55834574884 for osd.5 2024-05-08T08:27:40.512 DEBUG:teuthology.parallel:result is None 2024-05-08T08:27:40.512 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T08:27:40.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:27:40.853 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:40.853 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:27:40.865 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-05-08T08:27:40.450726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18458,"ondisk_log_size":18458,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379420,"kb_used_data":14896,"kb_used_omap":55,"kb_used_meta":364360,"kb_avail":754595300,"statfs":{"total":773094113280,"available":772705587200,"internally_reserved":0,"allocated":15253504,"data_stored":8539788,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":373105167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001551"},"pg_stats":[{"pgid":"3.1f","version":"137'740","reported_seq":5498,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053843+0000","last_change":"2024-05-08T08:27:13.130208+0000","last_active":"2024-05-08T08:27:38.053843+0000","last_peered":"2024-05-08T08:27:38.053843+0000","last_clean":"2024-05-08T08:27:38.053843+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:38.053843+0000","last_undegraded":"2024-05-08T08:27:38.053843+0000","last_fullsized":"2024-05-08T08:27:38.053843+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:52.764201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'796","reported_seq":3762,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230931+0000","last_change":"2024-05-08T08:27:28.179242+0000","last_active":"2024-05-08T08:27:37.230931+0000","last_peered":"2024-05-08T08:27:37.230931+0000","last_clean":"2024-05-08T08:27:37.230931+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:37.230931+0000","last_undegraded":"2024-05-08T08:27:37.230931+0000","last_fullsized":"2024-05-08T08:27:37.230931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:09.650651+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"144'562","reported_seq":2136,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415591+0000","last_change":"2024-05-08T08:27:30.000096+0000","last_active":"2024-05-08T08:27:37.415591+0000","last_peered":"2024-05-08T08:27:37.415591+0000","last_clean":"2024-05-08T08:27:37.415591+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:37.415591+0000","last_undegraded":"2024-05-08T08:27:37.415591+0000","last_fullsized":"2024-05-08T08:27:37.415591+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:17:13.519824+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"144'794","reported_seq":7674,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415540+0000","last_change":"2024-05-08T08:27:30.000107+0000","last_active":"2024-05-08T08:27:37.415540+0000","last_peered":"2024-05-08T08:27:37.415540+0000","last_clean":"2024-05-08T08:27:37.415540+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:37.415540+0000","last_undegraded":"2024-05-08T08:27:37.415540+0000","last_fullsized":"2024-05-08T08:27:37.415540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:45.406288+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"144'1100","reported_seq":2794,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054757+0000","last_change":"2024-05-08T08:27:23.244589+0000","last_active":"2024-05-08T08:27:38.054757+0000","last_peered":"2024-05-08T08:27:38.054757+0000","last_clean":"2024-05-08T08:27:38.054757+0000","last_became_active":"2024-05-08T08:27:23.151929+0000","last_became_peered":"2024-05-08T08:27:23.151929+0000","last_unstale":"2024-05-08T08:27:38.054757+0000","last_undegraded":"2024-05-08T08:27:38.054757+0000","last_fullsized":"2024-05-08T08:27:38.054757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.532972+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"144'465","reported_seq":12527,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232213+0000","last_change":"2024-05-08T08:27:23.298683+0000","last_active":"2024-05-08T08:27:37.232213+0000","last_peered":"2024-05-08T08:27:37.232213+0000","last_clean":"2024-05-08T08:27:37.232213+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:37.232213+0000","last_undegraded":"2024-05-08T08:27:37.232213+0000","last_fullsized":"2024-05-08T08:27:37.232213+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:15.541483+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'433","reported_seq":2347,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226292+0000","last_change":"2024-05-08T08:27:13.121389+0000","last_active":"2024-05-08T08:27:36.226292+0000","last_peered":"2024-05-08T08:27:36.226292+0000","last_clean":"2024-05-08T08:27:36.226292+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:36.226292+0000","last_undegraded":"2024-05-08T08:27:36.226292+0000","last_fullsized":"2024-05-08T08:27:36.226292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:50.144904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'409","reported_seq":12780,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230842+0000","last_change":"2024-05-08T08:27:28.179669+0000","last_active":"2024-05-08T08:27:37.230842+0000","last_peered":"2024-05-08T08:27:37.230842+0000","last_clean":"2024-05-08T08:27:37.230842+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:37.230842+0000","last_undegraded":"2024-05-08T08:27:37.230842+0000","last_fullsized":"2024-05-08T08:27:37.230842+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:58.353989+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2479,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232346+0000","last_change":"2024-05-08T08:27:26.164009+0000","last_active":"2024-05-08T08:27:37.232346+0000","last_peered":"2024-05-08T08:27:37.232346+0000","last_clean":"2024-05-08T08:27:37.232346+0000","last_became_active":"2024-05-08T08:27:26.162218+0000","last_became_peered":"2024-05-08T08:27:26.162218+0000","last_unstale":"2024-05-08T08:27:37.232346+0000","last_undegraded":"2024-05-08T08:27:37.232346+0000","last_fullsized":"2024-05-08T08:27:37.232346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:24.166081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"136'390","reported_seq":2124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415661+0000","last_change":"2024-05-08T08:27:29.996790+0000","last_active":"2024-05-08T08:27:37.415661+0000","last_peered":"2024-05-08T08:27:37.415661+0000","last_clean":"2024-05-08T08:27:37.415661+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:37.415661+0000","last_undegraded":"2024-05-08T08:27:37.415661+0000","last_fullsized":"2024-05-08T08:27:37.415661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:44.557422+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"130'424","reported_seq":4167,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226484+0000","last_change":"2024-05-08T08:27:14.994770+0000","last_active":"2024-05-08T08:27:36.226484+0000","last_peered":"2024-05-08T08:27:36.226484+0000","last_clean":"2024-05-08T08:27:36.226484+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:36.226484+0000","last_undegraded":"2024-05-08T08:27:36.226484+0000","last_fullsized":"2024-05-08T08:27:36.226484+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:18:01.277964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"144'509","reported_seq":1509,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232287+0000","last_change":"2024-05-08T08:27:23.281639+0000","last_active":"2024-05-08T08:27:37.232287+0000","last_peered":"2024-05-08T08:27:37.232287+0000","last_clean":"2024-05-08T08:27:37.232287+0000","last_became_active":"2024-05-08T08:27:23.146211+0000","last_became_peered":"2024-05-08T08:27:23.146211+0000","last_unstale":"2024-05-08T08:27:37.232287+0000","last_undegraded":"2024-05-08T08:27:37.232287+0000","last_fullsized":"2024-05-08T08:27:37.232287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:58.260716+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413392+0000","last_change":"2024-05-08T08:27:05.057403+0000","last_active":"2024-05-08T08:27:36.413392+0000","last_peered":"2024-05-08T08:27:36.413392+0000","last_clean":"2024-05-08T08:27:36.413392+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:36.413392+0000","last_undegraded":"2024-05-08T08:27:36.413392+0000","last_fullsized":"2024-05-08T08:27:36.413392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:06.889029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"136'424","reported_seq":9197,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415706+0000","last_change":"2024-05-08T08:27:29.996807+0000","last_active":"2024-05-08T08:27:37.415706+0000","last_peered":"2024-05-08T08:27:37.415706+0000","last_clean":"2024-05-08T08:27:37.415706+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:37.415706+0000","last_undegraded":"2024-05-08T08:27:37.415706+0000","last_fullsized":"2024-05-08T08:27:37.415706+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:02.089980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413509+0000","last_change":"2024-05-08T08:27:05.061031+0000","last_active":"2024-05-08T08:27:36.413509+0000","last_peered":"2024-05-08T08:27:36.413509+0000","last_clean":"2024-05-08T08:27:36.413509+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:36.413509+0000","last_undegraded":"2024-05-08T08:27:36.413509+0000","last_fullsized":"2024-05-08T08:27:36.413509+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:49:14.437090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"136'452","reported_seq":5709,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:28.180565+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.778235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":235,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232386+0000","last_change":"2024-05-08T08:27:25.160288+0000","last_active":"2024-05-08T08:27:37.232386+0000","last_peered":"2024-05-08T08:27:37.232386+0000","last_clean":"2024-05-08T08:27:37.232386+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:37.232386+0000","last_undegraded":"2024-05-08T08:27:37.232386+0000","last_fullsized":"2024-05-08T08:27:37.232386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:49.667882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'850","reported_seq":7956,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231057+0000","last_change":"2024-05-08T08:27:28.182287+0000","last_active":"2024-05-08T08:27:37.231057+0000","last_peered":"2024-05-08T08:27:37.231057+0000","last_clean":"2024-05-08T08:27:37.231057+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:37.231057+0000","last_undegraded":"2024-05-08T08:27:37.231057+0000","last_fullsized":"2024-05-08T08:27:37.231057+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:45.798327+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":296,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231165+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:27:37.231165+0000","last_peered":"2024-05-08T08:27:37.231165+0000","last_clean":"2024-05-08T08:27:37.231165+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:37.231165+0000","last_undegraded":"2024-05-08T08:27:37.231165+0000","last_fullsized":"2024-05-08T08:27:37.231165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'513","reported_seq":3310,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054297+0000","last_change":"2024-05-08T08:27:13.130200+0000","last_active":"2024-05-08T08:27:38.054297+0000","last_peered":"2024-05-08T08:27:38.054297+0000","last_clean":"2024-05-08T08:27:38.054297+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:38.054297+0000","last_undegraded":"2024-05-08T08:27:38.054297+0000","last_fullsized":"2024-05-08T08:27:38.054297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:44.403890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"144'1264","reported_seq":3214,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054556+0000","last_change":"2024-05-08T08:27:23.316843+0000","last_active":"2024-05-08T08:27:38.054556+0000","last_peered":"2024-05-08T08:27:38.054556+0000","last_clean":"2024-05-08T08:27:38.054556+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:38.054556+0000","last_undegraded":"2024-05-08T08:27:38.054556+0000","last_fullsized":"2024-05-08T08:27:38.054556+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:47.097836+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":450,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416093+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:27:36.416093+0000","last_peered":"2024-05-08T08:27:36.416093+0000","last_clean":"2024-05-08T08:27:36.416093+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:36.416093+0000","last_undegraded":"2024-05-08T08:27:36.416093+0000","last_fullsized":"2024-05-08T08:27:36.416093+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":232,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:05.059497+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:50:56.853131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"136'363","reported_seq":1899,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226181+0000","last_change":"2024-05-08T08:27:13.120989+0000","last_active":"2024-05-08T08:27:36.226181+0000","last_peered":"2024-05-08T08:27:36.226181+0000","last_clean":"2024-05-08T08:27:36.226181+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:36.226181+0000","last_undegraded":"2024-05-08T08:27:36.226181+0000","last_fullsized":"2024-05-08T08:27:36.226181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:32.583458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416150+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:27:36.416150+0000","last_peered":"2024-05-08T08:27:36.416150+0000","last_clean":"2024-05-08T08:27:36.416150+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:36.416150+0000","last_undegraded":"2024-05-08T08:27:36.416150+0000","last_fullsized":"2024-05-08T08:27:36.416150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"137'797","reported_seq":6168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231399+0000","last_change":"2024-05-08T08:27:13.158703+0000","last_active":"2024-05-08T08:27:37.231399+0000","last_peered":"2024-05-08T08:27:37.231399+0000","last_clean":"2024-05-08T08:27:37.231399+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:37.231399+0000","last_undegraded":"2024-05-08T08:27:37.231399+0000","last_fullsized":"2024-05-08T08:27:37.231399+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:44:42.443755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415518+0000","last_change":"2024-05-08T08:27:29.992916+0000","last_active":"2024-05-08T08:27:37.415518+0000","last_peered":"2024-05-08T08:27:37.415518+0000","last_clean":"2024-05-08T08:27:37.415518+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:37.415518+0000","last_undegraded":"2024-05-08T08:27:37.415518+0000","last_fullsized":"2024-05-08T08:27:37.415518+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:54.323209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1976,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232087+0000","last_change":"2024-05-08T08:27:26.167343+0000","last_active":"2024-05-08T08:27:37.232087+0000","last_peered":"2024-05-08T08:27:37.232087+0000","last_clean":"2024-05-08T08:27:37.232087+0000","last_became_active":"2024-05-08T08:27:26.165179+0000","last_became_peered":"2024-05-08T08:27:26.165179+0000","last_unstale":"2024-05-08T08:27:37.232087+0000","last_undegraded":"2024-05-08T08:27:37.232087+0000","last_fullsized":"2024-05-08T08:27:37.232087+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:23.290092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416094+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:27:36.416094+0000","last_peered":"2024-05-08T08:27:36.416094+0000","last_clean":"2024-05-08T08:27:36.416094+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:36.416094+0000","last_undegraded":"2024-05-08T08:27:36.416094+0000","last_fullsized":"2024-05-08T08:27:36.416094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":4994,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232007+0000","last_change":"2024-05-08T08:27:26.163935+0000","last_active":"2024-05-08T08:27:37.232007+0000","last_peered":"2024-05-08T08:27:37.232007+0000","last_clean":"2024-05-08T08:27:37.232007+0000","last_became_active":"2024-05-08T08:27:26.161990+0000","last_became_peered":"2024-05-08T08:27:26.161990+0000","last_unstale":"2024-05-08T08:27:37.232007+0000","last_undegraded":"2024-05-08T08:27:37.232007+0000","last_fullsized":"2024-05-08T08:27:37.232007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:25.301862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"144'353","reported_seq":1591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232115+0000","last_change":"2024-05-08T08:27:23.249997+0000","last_active":"2024-05-08T08:27:37.232115+0000","last_peered":"2024-05-08T08:27:37.232115+0000","last_clean":"2024-05-08T08:27:37.232115+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:37.232115+0000","last_undegraded":"2024-05-08T08:27:37.232115+0000","last_fullsized":"2024-05-08T08:27:37.232115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:45:17.630472+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"144'668","reported_seq":1986,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230934+0000","last_change":"2024-05-08T08:27:28.183953+0000","last_active":"2024-05-08T08:27:37.230934+0000","last_peered":"2024-05-08T08:27:37.230934+0000","last_clean":"2024-05-08T08:27:37.230934+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:37.230934+0000","last_undegraded":"2024-05-08T08:27:37.230934+0000","last_fullsized":"2024-05-08T08:27:37.230934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:13:51.258724+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'368","reported_seq":2574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230915+0000","last_change":"2024-05-08T08:27:28.179656+0000","last_active":"2024-05-08T08:27:37.230915+0000","last_peered":"2024-05-08T08:27:37.230915+0000","last_clean":"2024-05-08T08:27:37.230915+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:37.230915+0000","last_undegraded":"2024-05-08T08:27:37.230915+0000","last_fullsized":"2024-05-08T08:27:37.230915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:06:30.701923+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"130'373","reported_seq":4568,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415547+0000","last_change":"2024-05-08T08:27:29.996890+0000","last_active":"2024-05-08T08:27:37.415547+0000","last_peered":"2024-05-08T08:27:37.415547+0000","last_clean":"2024-05-08T08:27:37.415547+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:37.415547+0000","last_undegraded":"2024-05-08T08:27:37.415547+0000","last_fullsized":"2024-05-08T08:27:37.415547+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.108598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"136'373","reported_seq":1881,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416251+0000","last_change":"2024-05-08T08:27:13.115549+0000","last_active":"2024-05-08T08:27:36.416251+0000","last_peered":"2024-05-08T08:27:36.416251+0000","last_clean":"2024-05-08T08:27:36.416251+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:36.416251+0000","last_undegraded":"2024-05-08T08:27:36.416251+0000","last_fullsized":"2024-05-08T08:27:36.416251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:23.190217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'660","reported_seq":5045,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416292+0000","last_change":"2024-05-08T08:27:24.170963+0000","last_active":"2024-05-08T08:27:36.416292+0000","last_peered":"2024-05-08T08:27:36.416292+0000","last_clean":"2024-05-08T08:27:36.416292+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:36.416292+0000","last_undegraded":"2024-05-08T08:27:36.416292+0000","last_fullsized":"2024-05-08T08:27:36.416292+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:30.030308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"144'457","reported_seq":1925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416300+0000","last_change":"2024-05-08T08:27:23.187752+0000","last_active":"2024-05-08T08:27:36.416300+0000","last_peered":"2024-05-08T08:27:36.416300+0000","last_clean":"2024-05-08T08:27:36.416300+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:36.416300+0000","last_undegraded":"2024-05-08T08:27:36.416300+0000","last_fullsized":"2024-05-08T08:27:36.416300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:05.388094+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"137'698","reported_seq":3067,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413387+0000","last_change":"2024-05-08T08:27:13.117579+0000","last_active":"2024-05-08T08:27:36.413387+0000","last_peered":"2024-05-08T08:27:36.413387+0000","last_clean":"2024-05-08T08:27:36.413387+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:36.413387+0000","last_undegraded":"2024-05-08T08:27:36.413387+0000","last_fullsized":"2024-05-08T08:27:36.413387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:05.872143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"144'464","reported_seq":1874,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416108+0000","last_change":"2024-05-08T08:27:23.343798+0000","last_active":"2024-05-08T08:27:36.416108+0000","last_peered":"2024-05-08T08:27:36.416108+0000","last_clean":"2024-05-08T08:27:36.416108+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:36.416108+0000","last_undegraded":"2024-05-08T08:27:36.416108+0000","last_fullsized":"2024-05-08T08:27:36.416108+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:34.081134+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"133'375","reported_seq":3462,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053982+0000","last_change":"2024-05-08T08:27:13.129448+0000","last_active":"2024-05-08T08:27:38.053982+0000","last_peered":"2024-05-08T08:27:38.053982+0000","last_clean":"2024-05-08T08:27:38.053982+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:38.053982+0000","last_undegraded":"2024-05-08T08:27:38.053982+0000","last_fullsized":"2024-05-08T08:27:38.053982+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:51:57.502154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"144'706","reported_seq":4459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416199+0000","last_change":"2024-05-08T08:27:23.275749+0000","last_active":"2024-05-08T08:27:36.416199+0000","last_peered":"2024-05-08T08:27:36.416199+0000","last_clean":"2024-05-08T08:27:36.416199+0000","last_became_active":"2024-05-08T08:27:23.151481+0000","last_became_peered":"2024-05-08T08:27:23.151481+0000","last_unstale":"2024-05-08T08:27:36.416199+0000","last_undegraded":"2024-05-08T08:27:36.416199+0000","last_fullsized":"2024-05-08T08:27:36.416199+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:45.998918+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18341,"ondisk_log_size":18341,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":1508748,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35412,"kb_used_data":1860,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336428,"statfs":{"total":96636764160,"available":96600502272,"internally_reserved":0,"allocated":1904640,"data_stored":924386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234009,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43068,"kb_used_data":1904,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328772,"statfs":{"total":96636764160,"available":96592662528,"internally_reserved":0,"allocated":1949696,"data_stored":1508767,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:27:40.867 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:27:41.166 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:41.166 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:27:41.178 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-05-08T08:27:40.450726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18458,"ondisk_log_size":18458,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379420,"kb_used_data":14896,"kb_used_omap":55,"kb_used_meta":364360,"kb_avail":754595300,"statfs":{"total":773094113280,"available":772705587200,"internally_reserved":0,"allocated":15253504,"data_stored":8539788,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":373105167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001551"},"pg_stats":[{"pgid":"3.1f","version":"137'740","reported_seq":5498,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053843+0000","last_change":"2024-05-08T08:27:13.130208+0000","last_active":"2024-05-08T08:27:38.053843+0000","last_peered":"2024-05-08T08:27:38.053843+0000","last_clean":"2024-05-08T08:27:38.053843+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:38.053843+0000","last_undegraded":"2024-05-08T08:27:38.053843+0000","last_fullsized":"2024-05-08T08:27:38.053843+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:52.764201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'796","reported_seq":3762,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230931+0000","last_change":"2024-05-08T08:27:28.179242+0000","last_active":"2024-05-08T08:27:37.230931+0000","last_peered":"2024-05-08T08:27:37.230931+0000","last_clean":"2024-05-08T08:27:37.230931+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:37.230931+0000","last_undegraded":"2024-05-08T08:27:37.230931+0000","last_fullsized":"2024-05-08T08:27:37.230931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:09.650651+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"144'562","reported_seq":2136,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415591+0000","last_change":"2024-05-08T08:27:30.000096+0000","last_active":"2024-05-08T08:27:37.415591+0000","last_peered":"2024-05-08T08:27:37.415591+0000","last_clean":"2024-05-08T08:27:37.415591+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:37.415591+0000","last_undegraded":"2024-05-08T08:27:37.415591+0000","last_fullsized":"2024-05-08T08:27:37.415591+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:17:13.519824+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"144'794","reported_seq":7674,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415540+0000","last_change":"2024-05-08T08:27:30.000107+0000","last_active":"2024-05-08T08:27:37.415540+0000","last_peered":"2024-05-08T08:27:37.415540+0000","last_clean":"2024-05-08T08:27:37.415540+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:37.415540+0000","last_undegraded":"2024-05-08T08:27:37.415540+0000","last_fullsized":"2024-05-08T08:27:37.415540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:45.406288+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"144'1100","reported_seq":2794,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054757+0000","last_change":"2024-05-08T08:27:23.244589+0000","last_active":"2024-05-08T08:27:38.054757+0000","last_peered":"2024-05-08T08:27:38.054757+0000","last_clean":"2024-05-08T08:27:38.054757+0000","last_became_active":"2024-05-08T08:27:23.151929+0000","last_became_peered":"2024-05-08T08:27:23.151929+0000","last_unstale":"2024-05-08T08:27:38.054757+0000","last_undegraded":"2024-05-08T08:27:38.054757+0000","last_fullsized":"2024-05-08T08:27:38.054757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.532972+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"144'465","reported_seq":12527,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232213+0000","last_change":"2024-05-08T08:27:23.298683+0000","last_active":"2024-05-08T08:27:37.232213+0000","last_peered":"2024-05-08T08:27:37.232213+0000","last_clean":"2024-05-08T08:27:37.232213+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:37.232213+0000","last_undegraded":"2024-05-08T08:27:37.232213+0000","last_fullsized":"2024-05-08T08:27:37.232213+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:15.541483+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'433","reported_seq":2347,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226292+0000","last_change":"2024-05-08T08:27:13.121389+0000","last_active":"2024-05-08T08:27:36.226292+0000","last_peered":"2024-05-08T08:27:36.226292+0000","last_clean":"2024-05-08T08:27:36.226292+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:36.226292+0000","last_undegraded":"2024-05-08T08:27:36.226292+0000","last_fullsized":"2024-05-08T08:27:36.226292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:50.144904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'409","reported_seq":12780,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230842+0000","last_change":"2024-05-08T08:27:28.179669+0000","last_active":"2024-05-08T08:27:37.230842+0000","last_peered":"2024-05-08T08:27:37.230842+0000","last_clean":"2024-05-08T08:27:37.230842+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:37.230842+0000","last_undegraded":"2024-05-08T08:27:37.230842+0000","last_fullsized":"2024-05-08T08:27:37.230842+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:58.353989+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2479,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232346+0000","last_change":"2024-05-08T08:27:26.164009+0000","last_active":"2024-05-08T08:27:37.232346+0000","last_peered":"2024-05-08T08:27:37.232346+0000","last_clean":"2024-05-08T08:27:37.232346+0000","last_became_active":"2024-05-08T08:27:26.162218+0000","last_became_peered":"2024-05-08T08:27:26.162218+0000","last_unstale":"2024-05-08T08:27:37.232346+0000","last_undegraded":"2024-05-08T08:27:37.232346+0000","last_fullsized":"2024-05-08T08:27:37.232346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:24.166081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"136'390","reported_seq":2124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415661+0000","last_change":"2024-05-08T08:27:29.996790+0000","last_active":"2024-05-08T08:27:37.415661+0000","last_peered":"2024-05-08T08:27:37.415661+0000","last_clean":"2024-05-08T08:27:37.415661+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:37.415661+0000","last_undegraded":"2024-05-08T08:27:37.415661+0000","last_fullsized":"2024-05-08T08:27:37.415661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:44.557422+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"130'424","reported_seq":4167,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226484+0000","last_change":"2024-05-08T08:27:14.994770+0000","last_active":"2024-05-08T08:27:36.226484+0000","last_peered":"2024-05-08T08:27:36.226484+0000","last_clean":"2024-05-08T08:27:36.226484+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:36.226484+0000","last_undegraded":"2024-05-08T08:27:36.226484+0000","last_fullsized":"2024-05-08T08:27:36.226484+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:18:01.277964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"144'509","reported_seq":1509,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232287+0000","last_change":"2024-05-08T08:27:23.281639+0000","last_active":"2024-05-08T08:27:37.232287+0000","last_peered":"2024-05-08T08:27:37.232287+0000","last_clean":"2024-05-08T08:27:37.232287+0000","last_became_active":"2024-05-08T08:27:23.146211+0000","last_became_peered":"2024-05-08T08:27:23.146211+0000","last_unstale":"2024-05-08T08:27:37.232287+0000","last_undegraded":"2024-05-08T08:27:37.232287+0000","last_fullsized":"2024-05-08T08:27:37.232287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:58.260716+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413392+0000","last_change":"2024-05-08T08:27:05.057403+0000","last_active":"2024-05-08T08:27:36.413392+0000","last_peered":"2024-05-08T08:27:36.413392+0000","last_clean":"2024-05-08T08:27:36.413392+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:36.413392+0000","last_undegraded":"2024-05-08T08:27:36.413392+0000","last_fullsized":"2024-05-08T08:27:36.413392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:06.889029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"136'424","reported_seq":9197,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415706+0000","last_change":"2024-05-08T08:27:29.996807+0000","last_active":"2024-05-08T08:27:37.415706+0000","last_peered":"2024-05-08T08:27:37.415706+0000","last_clean":"2024-05-08T08:27:37.415706+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:37.415706+0000","last_undegraded":"2024-05-08T08:27:37.415706+0000","last_fullsized":"2024-05-08T08:27:37.415706+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:02.089980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413509+0000","last_change":"2024-05-08T08:27:05.061031+0000","last_active":"2024-05-08T08:27:36.413509+0000","last_peered":"2024-05-08T08:27:36.413509+0000","last_clean":"2024-05-08T08:27:36.413509+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:36.413509+0000","last_undegraded":"2024-05-08T08:27:36.413509+0000","last_fullsized":"2024-05-08T08:27:36.413509+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:49:14.437090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"136'452","reported_seq":5709,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:28.180565+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.778235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":235,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232386+0000","last_change":"2024-05-08T08:27:25.160288+0000","last_active":"2024-05-08T08:27:37.232386+0000","last_peered":"2024-05-08T08:27:37.232386+0000","last_clean":"2024-05-08T08:27:37.232386+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:37.232386+0000","last_undegraded":"2024-05-08T08:27:37.232386+0000","last_fullsized":"2024-05-08T08:27:37.232386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:49.667882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'850","reported_seq":7956,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231057+0000","last_change":"2024-05-08T08:27:28.182287+0000","last_active":"2024-05-08T08:27:37.231057+0000","last_peered":"2024-05-08T08:27:37.231057+0000","last_clean":"2024-05-08T08:27:37.231057+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:37.231057+0000","last_undegraded":"2024-05-08T08:27:37.231057+0000","last_fullsized":"2024-05-08T08:27:37.231057+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:45.798327+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":296,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231165+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:27:37.231165+0000","last_peered":"2024-05-08T08:27:37.231165+0000","last_clean":"2024-05-08T08:27:37.231165+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:37.231165+0000","last_undegraded":"2024-05-08T08:27:37.231165+0000","last_fullsized":"2024-05-08T08:27:37.231165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'513","reported_seq":3310,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054297+0000","last_change":"2024-05-08T08:27:13.130200+0000","last_active":"2024-05-08T08:27:38.054297+0000","last_peered":"2024-05-08T08:27:38.054297+0000","last_clean":"2024-05-08T08:27:38.054297+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:38.054297+0000","last_undegraded":"2024-05-08T08:27:38.054297+0000","last_fullsized":"2024-05-08T08:27:38.054297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:44.403890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"144'1264","reported_seq":3214,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054556+0000","last_change":"2024-05-08T08:27:23.316843+0000","last_active":"2024-05-08T08:27:38.054556+0000","last_peered":"2024-05-08T08:27:38.054556+0000","last_clean":"2024-05-08T08:27:38.054556+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:38.054556+0000","last_undegraded":"2024-05-08T08:27:38.054556+0000","last_fullsized":"2024-05-08T08:27:38.054556+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:47.097836+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":450,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416093+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:27:36.416093+0000","last_peered":"2024-05-08T08:27:36.416093+0000","last_clean":"2024-05-08T08:27:36.416093+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:36.416093+0000","last_undegraded":"2024-05-08T08:27:36.416093+0000","last_fullsized":"2024-05-08T08:27:36.416093+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":232,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:05.059497+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:50:56.853131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"136'363","reported_seq":1899,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226181+0000","last_change":"2024-05-08T08:27:13.120989+0000","last_active":"2024-05-08T08:27:36.226181+0000","last_peered":"2024-05-08T08:27:36.226181+0000","last_clean":"2024-05-08T08:27:36.226181+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:36.226181+0000","last_undegraded":"2024-05-08T08:27:36.226181+0000","last_fullsized":"2024-05-08T08:27:36.226181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:32.583458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416150+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:27:36.416150+0000","last_peered":"2024-05-08T08:27:36.416150+0000","last_clean":"2024-05-08T08:27:36.416150+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:36.416150+0000","last_undegraded":"2024-05-08T08:27:36.416150+0000","last_fullsized":"2024-05-08T08:27:36.416150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"137'797","reported_seq":6168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231399+0000","last_change":"2024-05-08T08:27:13.158703+0000","last_active":"2024-05-08T08:27:37.231399+0000","last_peered":"2024-05-08T08:27:37.231399+0000","last_clean":"2024-05-08T08:27:37.231399+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:37.231399+0000","last_undegraded":"2024-05-08T08:27:37.231399+0000","last_fullsized":"2024-05-08T08:27:37.231399+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:44:42.443755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415518+0000","last_change":"2024-05-08T08:27:29.992916+0000","last_active":"2024-05-08T08:27:37.415518+0000","last_peered":"2024-05-08T08:27:37.415518+0000","last_clean":"2024-05-08T08:27:37.415518+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:37.415518+0000","last_undegraded":"2024-05-08T08:27:37.415518+0000","last_fullsized":"2024-05-08T08:27:37.415518+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:54.323209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1976,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232087+0000","last_change":"2024-05-08T08:27:26.167343+0000","last_active":"2024-05-08T08:27:37.232087+0000","last_peered":"2024-05-08T08:27:37.232087+0000","last_clean":"2024-05-08T08:27:37.232087+0000","last_became_active":"2024-05-08T08:27:26.165179+0000","last_became_peered":"2024-05-08T08:27:26.165179+0000","last_unstale":"2024-05-08T08:27:37.232087+0000","last_undegraded":"2024-05-08T08:27:37.232087+0000","last_fullsized":"2024-05-08T08:27:37.232087+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:23.290092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416094+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:27:36.416094+0000","last_peered":"2024-05-08T08:27:36.416094+0000","last_clean":"2024-05-08T08:27:36.416094+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:36.416094+0000","last_undegraded":"2024-05-08T08:27:36.416094+0000","last_fullsized":"2024-05-08T08:27:36.416094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":4994,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232007+0000","last_change":"2024-05-08T08:27:26.163935+0000","last_active":"2024-05-08T08:27:37.232007+0000","last_peered":"2024-05-08T08:27:37.232007+0000","last_clean":"2024-05-08T08:27:37.232007+0000","last_became_active":"2024-05-08T08:27:26.161990+0000","last_became_peered":"2024-05-08T08:27:26.161990+0000","last_unstale":"2024-05-08T08:27:37.232007+0000","last_undegraded":"2024-05-08T08:27:37.232007+0000","last_fullsized":"2024-05-08T08:27:37.232007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:25.301862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"144'353","reported_seq":1591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232115+0000","last_change":"2024-05-08T08:27:23.249997+0000","last_active":"2024-05-08T08:27:37.232115+0000","last_peered":"2024-05-08T08:27:37.232115+0000","last_clean":"2024-05-08T08:27:37.232115+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:37.232115+0000","last_undegraded":"2024-05-08T08:27:37.232115+0000","last_fullsized":"2024-05-08T08:27:37.232115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:45:17.630472+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"144'668","reported_seq":1986,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230934+0000","last_change":"2024-05-08T08:27:28.183953+0000","last_active":"2024-05-08T08:27:37.230934+0000","last_peered":"2024-05-08T08:27:37.230934+0000","last_clean":"2024-05-08T08:27:37.230934+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:37.230934+0000","last_undegraded":"2024-05-08T08:27:37.230934+0000","last_fullsized":"2024-05-08T08:27:37.230934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:13:51.258724+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'368","reported_seq":2574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230915+0000","last_change":"2024-05-08T08:27:28.179656+0000","last_active":"2024-05-08T08:27:37.230915+0000","last_peered":"2024-05-08T08:27:37.230915+0000","last_clean":"2024-05-08T08:27:37.230915+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:37.230915+0000","last_undegraded":"2024-05-08T08:27:37.230915+0000","last_fullsized":"2024-05-08T08:27:37.230915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:06:30.701923+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"130'373","reported_seq":4568,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415547+0000","last_change":"2024-05-08T08:27:29.996890+0000","last_active":"2024-05-08T08:27:37.415547+0000","last_peered":"2024-05-08T08:27:37.415547+0000","last_clean":"2024-05-08T08:27:37.415547+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:37.415547+0000","last_undegraded":"2024-05-08T08:27:37.415547+0000","last_fullsized":"2024-05-08T08:27:37.415547+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.108598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"136'373","reported_seq":1881,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416251+0000","last_change":"2024-05-08T08:27:13.115549+0000","last_active":"2024-05-08T08:27:36.416251+0000","last_peered":"2024-05-08T08:27:36.416251+0000","last_clean":"2024-05-08T08:27:36.416251+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:36.416251+0000","last_undegraded":"2024-05-08T08:27:36.416251+0000","last_fullsized":"2024-05-08T08:27:36.416251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:23.190217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'660","reported_seq":5045,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416292+0000","last_change":"2024-05-08T08:27:24.170963+0000","last_active":"2024-05-08T08:27:36.416292+0000","last_peered":"2024-05-08T08:27:36.416292+0000","last_clean":"2024-05-08T08:27:36.416292+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:36.416292+0000","last_undegraded":"2024-05-08T08:27:36.416292+0000","last_fullsized":"2024-05-08T08:27:36.416292+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:30.030308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"144'457","reported_seq":1925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416300+0000","last_change":"2024-05-08T08:27:23.187752+0000","last_active":"2024-05-08T08:27:36.416300+0000","last_peered":"2024-05-08T08:27:36.416300+0000","last_clean":"2024-05-08T08:27:36.416300+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:36.416300+0000","last_undegraded":"2024-05-08T08:27:36.416300+0000","last_fullsized":"2024-05-08T08:27:36.416300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:05.388094+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"137'698","reported_seq":3067,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413387+0000","last_change":"2024-05-08T08:27:13.117579+0000","last_active":"2024-05-08T08:27:36.413387+0000","last_peered":"2024-05-08T08:27:36.413387+0000","last_clean":"2024-05-08T08:27:36.413387+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:36.413387+0000","last_undegraded":"2024-05-08T08:27:36.413387+0000","last_fullsized":"2024-05-08T08:27:36.413387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:05.872143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"144'464","reported_seq":1874,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416108+0000","last_change":"2024-05-08T08:27:23.343798+0000","last_active":"2024-05-08T08:27:36.416108+0000","last_peered":"2024-05-08T08:27:36.416108+0000","last_clean":"2024-05-08T08:27:36.416108+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:36.416108+0000","last_undegraded":"2024-05-08T08:27:36.416108+0000","last_fullsized":"2024-05-08T08:27:36.416108+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:34.081134+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"133'375","reported_seq":3462,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053982+0000","last_change":"2024-05-08T08:27:13.129448+0000","last_active":"2024-05-08T08:27:38.053982+0000","last_peered":"2024-05-08T08:27:38.053982+0000","last_clean":"2024-05-08T08:27:38.053982+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:38.053982+0000","last_undegraded":"2024-05-08T08:27:38.053982+0000","last_fullsized":"2024-05-08T08:27:38.053982+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:51:57.502154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"144'706","reported_seq":4459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416199+0000","last_change":"2024-05-08T08:27:23.275749+0000","last_active":"2024-05-08T08:27:36.416199+0000","last_peered":"2024-05-08T08:27:36.416199+0000","last_clean":"2024-05-08T08:27:36.416199+0000","last_became_active":"2024-05-08T08:27:23.151481+0000","last_became_peered":"2024-05-08T08:27:23.151481+0000","last_unstale":"2024-05-08T08:27:36.416199+0000","last_undegraded":"2024-05-08T08:27:36.416199+0000","last_fullsized":"2024-05-08T08:27:36.416199+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:45.998918+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18341,"ondisk_log_size":18341,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":1508748,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35412,"kb_used_data":1860,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336428,"statfs":{"total":96636764160,"available":96600502272,"internally_reserved":0,"allocated":1904640,"data_stored":924386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234009,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43068,"kb_used_data":1904,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328772,"statfs":{"total":96636764160,"available":96592662528,"internally_reserved":0,"allocated":1949696,"data_stored":1508767,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:27:41.180 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T08:27:41.180 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-05-08T08:27:41.499 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T08:27:41.508 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T08:27:41.509 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:27:41.808 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:41.808 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:27:41.820 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-05-08T08:27:40.450726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18458,"ondisk_log_size":18458,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379420,"kb_used_data":14896,"kb_used_omap":55,"kb_used_meta":364360,"kb_avail":754595300,"statfs":{"total":773094113280,"available":772705587200,"internally_reserved":0,"allocated":15253504,"data_stored":8539788,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":373105167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001551"},"pg_stats":[{"pgid":"3.1f","version":"137'740","reported_seq":5498,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053843+0000","last_change":"2024-05-08T08:27:13.130208+0000","last_active":"2024-05-08T08:27:38.053843+0000","last_peered":"2024-05-08T08:27:38.053843+0000","last_clean":"2024-05-08T08:27:38.053843+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:38.053843+0000","last_undegraded":"2024-05-08T08:27:38.053843+0000","last_fullsized":"2024-05-08T08:27:38.053843+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:52.764201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'796","reported_seq":3762,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230931+0000","last_change":"2024-05-08T08:27:28.179242+0000","last_active":"2024-05-08T08:27:37.230931+0000","last_peered":"2024-05-08T08:27:37.230931+0000","last_clean":"2024-05-08T08:27:37.230931+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:37.230931+0000","last_undegraded":"2024-05-08T08:27:37.230931+0000","last_fullsized":"2024-05-08T08:27:37.230931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:09.650651+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"144'562","reported_seq":2136,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415591+0000","last_change":"2024-05-08T08:27:30.000096+0000","last_active":"2024-05-08T08:27:37.415591+0000","last_peered":"2024-05-08T08:27:37.415591+0000","last_clean":"2024-05-08T08:27:37.415591+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:37.415591+0000","last_undegraded":"2024-05-08T08:27:37.415591+0000","last_fullsized":"2024-05-08T08:27:37.415591+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:17:13.519824+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"144'794","reported_seq":7674,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415540+0000","last_change":"2024-05-08T08:27:30.000107+0000","last_active":"2024-05-08T08:27:37.415540+0000","last_peered":"2024-05-08T08:27:37.415540+0000","last_clean":"2024-05-08T08:27:37.415540+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:37.415540+0000","last_undegraded":"2024-05-08T08:27:37.415540+0000","last_fullsized":"2024-05-08T08:27:37.415540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:45.406288+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"144'1100","reported_seq":2794,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054757+0000","last_change":"2024-05-08T08:27:23.244589+0000","last_active":"2024-05-08T08:27:38.054757+0000","last_peered":"2024-05-08T08:27:38.054757+0000","last_clean":"2024-05-08T08:27:38.054757+0000","last_became_active":"2024-05-08T08:27:23.151929+0000","last_became_peered":"2024-05-08T08:27:23.151929+0000","last_unstale":"2024-05-08T08:27:38.054757+0000","last_undegraded":"2024-05-08T08:27:38.054757+0000","last_fullsized":"2024-05-08T08:27:38.054757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.532972+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"144'465","reported_seq":12527,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232213+0000","last_change":"2024-05-08T08:27:23.298683+0000","last_active":"2024-05-08T08:27:37.232213+0000","last_peered":"2024-05-08T08:27:37.232213+0000","last_clean":"2024-05-08T08:27:37.232213+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:37.232213+0000","last_undegraded":"2024-05-08T08:27:37.232213+0000","last_fullsized":"2024-05-08T08:27:37.232213+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:15.541483+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'433","reported_seq":2347,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226292+0000","last_change":"2024-05-08T08:27:13.121389+0000","last_active":"2024-05-08T08:27:36.226292+0000","last_peered":"2024-05-08T08:27:36.226292+0000","last_clean":"2024-05-08T08:27:36.226292+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:36.226292+0000","last_undegraded":"2024-05-08T08:27:36.226292+0000","last_fullsized":"2024-05-08T08:27:36.226292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:50.144904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'409","reported_seq":12780,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230842+0000","last_change":"2024-05-08T08:27:28.179669+0000","last_active":"2024-05-08T08:27:37.230842+0000","last_peered":"2024-05-08T08:27:37.230842+0000","last_clean":"2024-05-08T08:27:37.230842+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:37.230842+0000","last_undegraded":"2024-05-08T08:27:37.230842+0000","last_fullsized":"2024-05-08T08:27:37.230842+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:58.353989+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2479,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232346+0000","last_change":"2024-05-08T08:27:26.164009+0000","last_active":"2024-05-08T08:27:37.232346+0000","last_peered":"2024-05-08T08:27:37.232346+0000","last_clean":"2024-05-08T08:27:37.232346+0000","last_became_active":"2024-05-08T08:27:26.162218+0000","last_became_peered":"2024-05-08T08:27:26.162218+0000","last_unstale":"2024-05-08T08:27:37.232346+0000","last_undegraded":"2024-05-08T08:27:37.232346+0000","last_fullsized":"2024-05-08T08:27:37.232346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:24.166081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"136'390","reported_seq":2124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415661+0000","last_change":"2024-05-08T08:27:29.996790+0000","last_active":"2024-05-08T08:27:37.415661+0000","last_peered":"2024-05-08T08:27:37.415661+0000","last_clean":"2024-05-08T08:27:37.415661+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:37.415661+0000","last_undegraded":"2024-05-08T08:27:37.415661+0000","last_fullsized":"2024-05-08T08:27:37.415661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:44.557422+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"130'424","reported_seq":4167,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226484+0000","last_change":"2024-05-08T08:27:14.994770+0000","last_active":"2024-05-08T08:27:36.226484+0000","last_peered":"2024-05-08T08:27:36.226484+0000","last_clean":"2024-05-08T08:27:36.226484+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:36.226484+0000","last_undegraded":"2024-05-08T08:27:36.226484+0000","last_fullsized":"2024-05-08T08:27:36.226484+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:18:01.277964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"144'509","reported_seq":1509,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232287+0000","last_change":"2024-05-08T08:27:23.281639+0000","last_active":"2024-05-08T08:27:37.232287+0000","last_peered":"2024-05-08T08:27:37.232287+0000","last_clean":"2024-05-08T08:27:37.232287+0000","last_became_active":"2024-05-08T08:27:23.146211+0000","last_became_peered":"2024-05-08T08:27:23.146211+0000","last_unstale":"2024-05-08T08:27:37.232287+0000","last_undegraded":"2024-05-08T08:27:37.232287+0000","last_fullsized":"2024-05-08T08:27:37.232287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:58.260716+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413392+0000","last_change":"2024-05-08T08:27:05.057403+0000","last_active":"2024-05-08T08:27:36.413392+0000","last_peered":"2024-05-08T08:27:36.413392+0000","last_clean":"2024-05-08T08:27:36.413392+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:36.413392+0000","last_undegraded":"2024-05-08T08:27:36.413392+0000","last_fullsized":"2024-05-08T08:27:36.413392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:06.889029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"136'424","reported_seq":9197,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415706+0000","last_change":"2024-05-08T08:27:29.996807+0000","last_active":"2024-05-08T08:27:37.415706+0000","last_peered":"2024-05-08T08:27:37.415706+0000","last_clean":"2024-05-08T08:27:37.415706+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:37.415706+0000","last_undegraded":"2024-05-08T08:27:37.415706+0000","last_fullsized":"2024-05-08T08:27:37.415706+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:02.089980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413509+0000","last_change":"2024-05-08T08:27:05.061031+0000","last_active":"2024-05-08T08:27:36.413509+0000","last_peered":"2024-05-08T08:27:36.413509+0000","last_clean":"2024-05-08T08:27:36.413509+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:36.413509+0000","last_undegraded":"2024-05-08T08:27:36.413509+0000","last_fullsized":"2024-05-08T08:27:36.413509+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:49:14.437090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"136'452","reported_seq":5709,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:28.180565+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.778235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":235,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232386+0000","last_change":"2024-05-08T08:27:25.160288+0000","last_active":"2024-05-08T08:27:37.232386+0000","last_peered":"2024-05-08T08:27:37.232386+0000","last_clean":"2024-05-08T08:27:37.232386+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:37.232386+0000","last_undegraded":"2024-05-08T08:27:37.232386+0000","last_fullsized":"2024-05-08T08:27:37.232386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:49.667882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'850","reported_seq":7956,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231057+0000","last_change":"2024-05-08T08:27:28.182287+0000","last_active":"2024-05-08T08:27:37.231057+0000","last_peered":"2024-05-08T08:27:37.231057+0000","last_clean":"2024-05-08T08:27:37.231057+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:37.231057+0000","last_undegraded":"2024-05-08T08:27:37.231057+0000","last_fullsized":"2024-05-08T08:27:37.231057+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:45.798327+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":296,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231165+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:27:37.231165+0000","last_peered":"2024-05-08T08:27:37.231165+0000","last_clean":"2024-05-08T08:27:37.231165+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:37.231165+0000","last_undegraded":"2024-05-08T08:27:37.231165+0000","last_fullsized":"2024-05-08T08:27:37.231165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'513","reported_seq":3310,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054297+0000","last_change":"2024-05-08T08:27:13.130200+0000","last_active":"2024-05-08T08:27:38.054297+0000","last_peered":"2024-05-08T08:27:38.054297+0000","last_clean":"2024-05-08T08:27:38.054297+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:38.054297+0000","last_undegraded":"2024-05-08T08:27:38.054297+0000","last_fullsized":"2024-05-08T08:27:38.054297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:44.403890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"144'1264","reported_seq":3214,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054556+0000","last_change":"2024-05-08T08:27:23.316843+0000","last_active":"2024-05-08T08:27:38.054556+0000","last_peered":"2024-05-08T08:27:38.054556+0000","last_clean":"2024-05-08T08:27:38.054556+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:38.054556+0000","last_undegraded":"2024-05-08T08:27:38.054556+0000","last_fullsized":"2024-05-08T08:27:38.054556+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:47.097836+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":450,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416093+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:27:36.416093+0000","last_peered":"2024-05-08T08:27:36.416093+0000","last_clean":"2024-05-08T08:27:36.416093+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:36.416093+0000","last_undegraded":"2024-05-08T08:27:36.416093+0000","last_fullsized":"2024-05-08T08:27:36.416093+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":232,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:05.059497+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:50:56.853131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"136'363","reported_seq":1899,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226181+0000","last_change":"2024-05-08T08:27:13.120989+0000","last_active":"2024-05-08T08:27:36.226181+0000","last_peered":"2024-05-08T08:27:36.226181+0000","last_clean":"2024-05-08T08:27:36.226181+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:36.226181+0000","last_undegraded":"2024-05-08T08:27:36.226181+0000","last_fullsized":"2024-05-08T08:27:36.226181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:32.583458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416150+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:27:36.416150+0000","last_peered":"2024-05-08T08:27:36.416150+0000","last_clean":"2024-05-08T08:27:36.416150+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:36.416150+0000","last_undegraded":"2024-05-08T08:27:36.416150+0000","last_fullsized":"2024-05-08T08:27:36.416150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"137'797","reported_seq":6168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231399+0000","last_change":"2024-05-08T08:27:13.158703+0000","last_active":"2024-05-08T08:27:37.231399+0000","last_peered":"2024-05-08T08:27:37.231399+0000","last_clean":"2024-05-08T08:27:37.231399+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:37.231399+0000","last_undegraded":"2024-05-08T08:27:37.231399+0000","last_fullsized":"2024-05-08T08:27:37.231399+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:44:42.443755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415518+0000","last_change":"2024-05-08T08:27:29.992916+0000","last_active":"2024-05-08T08:27:37.415518+0000","last_peered":"2024-05-08T08:27:37.415518+0000","last_clean":"2024-05-08T08:27:37.415518+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:37.415518+0000","last_undegraded":"2024-05-08T08:27:37.415518+0000","last_fullsized":"2024-05-08T08:27:37.415518+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:54.323209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1976,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232087+0000","last_change":"2024-05-08T08:27:26.167343+0000","last_active":"2024-05-08T08:27:37.232087+0000","last_peered":"2024-05-08T08:27:37.232087+0000","last_clean":"2024-05-08T08:27:37.232087+0000","last_became_active":"2024-05-08T08:27:26.165179+0000","last_became_peered":"2024-05-08T08:27:26.165179+0000","last_unstale":"2024-05-08T08:27:37.232087+0000","last_undegraded":"2024-05-08T08:27:37.232087+0000","last_fullsized":"2024-05-08T08:27:37.232087+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:23.290092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416094+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:27:36.416094+0000","last_peered":"2024-05-08T08:27:36.416094+0000","last_clean":"2024-05-08T08:27:36.416094+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:36.416094+0000","last_undegraded":"2024-05-08T08:27:36.416094+0000","last_fullsized":"2024-05-08T08:27:36.416094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":4994,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232007+0000","last_change":"2024-05-08T08:27:26.163935+0000","last_active":"2024-05-08T08:27:37.232007+0000","last_peered":"2024-05-08T08:27:37.232007+0000","last_clean":"2024-05-08T08:27:37.232007+0000","last_became_active":"2024-05-08T08:27:26.161990+0000","last_became_peered":"2024-05-08T08:27:26.161990+0000","last_unstale":"2024-05-08T08:27:37.232007+0000","last_undegraded":"2024-05-08T08:27:37.232007+0000","last_fullsized":"2024-05-08T08:27:37.232007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:25.301862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"144'353","reported_seq":1591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232115+0000","last_change":"2024-05-08T08:27:23.249997+0000","last_active":"2024-05-08T08:27:37.232115+0000","last_peered":"2024-05-08T08:27:37.232115+0000","last_clean":"2024-05-08T08:27:37.232115+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:37.232115+0000","last_undegraded":"2024-05-08T08:27:37.232115+0000","last_fullsized":"2024-05-08T08:27:37.232115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:45:17.630472+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"144'668","reported_seq":1986,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230934+0000","last_change":"2024-05-08T08:27:28.183953+0000","last_active":"2024-05-08T08:27:37.230934+0000","last_peered":"2024-05-08T08:27:37.230934+0000","last_clean":"2024-05-08T08:27:37.230934+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:37.230934+0000","last_undegraded":"2024-05-08T08:27:37.230934+0000","last_fullsized":"2024-05-08T08:27:37.230934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:13:51.258724+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'368","reported_seq":2574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230915+0000","last_change":"2024-05-08T08:27:28.179656+0000","last_active":"2024-05-08T08:27:37.230915+0000","last_peered":"2024-05-08T08:27:37.230915+0000","last_clean":"2024-05-08T08:27:37.230915+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:37.230915+0000","last_undegraded":"2024-05-08T08:27:37.230915+0000","last_fullsized":"2024-05-08T08:27:37.230915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:06:30.701923+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"130'373","reported_seq":4568,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415547+0000","last_change":"2024-05-08T08:27:29.996890+0000","last_active":"2024-05-08T08:27:37.415547+0000","last_peered":"2024-05-08T08:27:37.415547+0000","last_clean":"2024-05-08T08:27:37.415547+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:37.415547+0000","last_undegraded":"2024-05-08T08:27:37.415547+0000","last_fullsized":"2024-05-08T08:27:37.415547+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.108598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"136'373","reported_seq":1881,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416251+0000","last_change":"2024-05-08T08:27:13.115549+0000","last_active":"2024-05-08T08:27:36.416251+0000","last_peered":"2024-05-08T08:27:36.416251+0000","last_clean":"2024-05-08T08:27:36.416251+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:36.416251+0000","last_undegraded":"2024-05-08T08:27:36.416251+0000","last_fullsized":"2024-05-08T08:27:36.416251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:23.190217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'660","reported_seq":5045,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416292+0000","last_change":"2024-05-08T08:27:24.170963+0000","last_active":"2024-05-08T08:27:36.416292+0000","last_peered":"2024-05-08T08:27:36.416292+0000","last_clean":"2024-05-08T08:27:36.416292+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:36.416292+0000","last_undegraded":"2024-05-08T08:27:36.416292+0000","last_fullsized":"2024-05-08T08:27:36.416292+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:30.030308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"144'457","reported_seq":1925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416300+0000","last_change":"2024-05-08T08:27:23.187752+0000","last_active":"2024-05-08T08:27:36.416300+0000","last_peered":"2024-05-08T08:27:36.416300+0000","last_clean":"2024-05-08T08:27:36.416300+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:36.416300+0000","last_undegraded":"2024-05-08T08:27:36.416300+0000","last_fullsized":"2024-05-08T08:27:36.416300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:05.388094+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"137'698","reported_seq":3067,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413387+0000","last_change":"2024-05-08T08:27:13.117579+0000","last_active":"2024-05-08T08:27:36.413387+0000","last_peered":"2024-05-08T08:27:36.413387+0000","last_clean":"2024-05-08T08:27:36.413387+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:36.413387+0000","last_undegraded":"2024-05-08T08:27:36.413387+0000","last_fullsized":"2024-05-08T08:27:36.413387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:05.872143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"144'464","reported_seq":1874,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416108+0000","last_change":"2024-05-08T08:27:23.343798+0000","last_active":"2024-05-08T08:27:36.416108+0000","last_peered":"2024-05-08T08:27:36.416108+0000","last_clean":"2024-05-08T08:27:36.416108+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:36.416108+0000","last_undegraded":"2024-05-08T08:27:36.416108+0000","last_fullsized":"2024-05-08T08:27:36.416108+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:34.081134+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"133'375","reported_seq":3462,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053982+0000","last_change":"2024-05-08T08:27:13.129448+0000","last_active":"2024-05-08T08:27:38.053982+0000","last_peered":"2024-05-08T08:27:38.053982+0000","last_clean":"2024-05-08T08:27:38.053982+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:38.053982+0000","last_undegraded":"2024-05-08T08:27:38.053982+0000","last_fullsized":"2024-05-08T08:27:38.053982+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:51:57.502154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"144'706","reported_seq":4459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416199+0000","last_change":"2024-05-08T08:27:23.275749+0000","last_active":"2024-05-08T08:27:36.416199+0000","last_peered":"2024-05-08T08:27:36.416199+0000","last_clean":"2024-05-08T08:27:36.416199+0000","last_became_active":"2024-05-08T08:27:23.151481+0000","last_became_peered":"2024-05-08T08:27:23.151481+0000","last_unstale":"2024-05-08T08:27:36.416199+0000","last_undegraded":"2024-05-08T08:27:36.416199+0000","last_fullsized":"2024-05-08T08:27:36.416199+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:45.998918+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18341,"ondisk_log_size":18341,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":1508748,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35412,"kb_used_data":1860,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336428,"statfs":{"total":96636764160,"available":96600502272,"internally_reserved":0,"allocated":1904640,"data_stored":924386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234009,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43068,"kb_used_data":1904,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328772,"statfs":{"total":96636764160,"available":96592662528,"internally_reserved":0,"allocated":1949696,"data_stored":1508767,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:27:41.822 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:27:42.121 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:42.122 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:27:42.133 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-05-08T08:27:40.450726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18458,"ondisk_log_size":18458,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379420,"kb_used_data":14896,"kb_used_omap":55,"kb_used_meta":364360,"kb_avail":754595300,"statfs":{"total":773094113280,"available":772705587200,"internally_reserved":0,"allocated":15253504,"data_stored":8539788,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":373105167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001551"},"pg_stats":[{"pgid":"3.1f","version":"137'740","reported_seq":5498,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053843+0000","last_change":"2024-05-08T08:27:13.130208+0000","last_active":"2024-05-08T08:27:38.053843+0000","last_peered":"2024-05-08T08:27:38.053843+0000","last_clean":"2024-05-08T08:27:38.053843+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:38.053843+0000","last_undegraded":"2024-05-08T08:27:38.053843+0000","last_fullsized":"2024-05-08T08:27:38.053843+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:52.764201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'796","reported_seq":3762,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230931+0000","last_change":"2024-05-08T08:27:28.179242+0000","last_active":"2024-05-08T08:27:37.230931+0000","last_peered":"2024-05-08T08:27:37.230931+0000","last_clean":"2024-05-08T08:27:37.230931+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:37.230931+0000","last_undegraded":"2024-05-08T08:27:37.230931+0000","last_fullsized":"2024-05-08T08:27:37.230931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:09.650651+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"144'562","reported_seq":2136,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415591+0000","last_change":"2024-05-08T08:27:30.000096+0000","last_active":"2024-05-08T08:27:37.415591+0000","last_peered":"2024-05-08T08:27:37.415591+0000","last_clean":"2024-05-08T08:27:37.415591+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:37.415591+0000","last_undegraded":"2024-05-08T08:27:37.415591+0000","last_fullsized":"2024-05-08T08:27:37.415591+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:17:13.519824+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"144'794","reported_seq":7674,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415540+0000","last_change":"2024-05-08T08:27:30.000107+0000","last_active":"2024-05-08T08:27:37.415540+0000","last_peered":"2024-05-08T08:27:37.415540+0000","last_clean":"2024-05-08T08:27:37.415540+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:37.415540+0000","last_undegraded":"2024-05-08T08:27:37.415540+0000","last_fullsized":"2024-05-08T08:27:37.415540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:45.406288+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"144'1100","reported_seq":2794,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054757+0000","last_change":"2024-05-08T08:27:23.244589+0000","last_active":"2024-05-08T08:27:38.054757+0000","last_peered":"2024-05-08T08:27:38.054757+0000","last_clean":"2024-05-08T08:27:38.054757+0000","last_became_active":"2024-05-08T08:27:23.151929+0000","last_became_peered":"2024-05-08T08:27:23.151929+0000","last_unstale":"2024-05-08T08:27:38.054757+0000","last_undegraded":"2024-05-08T08:27:38.054757+0000","last_fullsized":"2024-05-08T08:27:38.054757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.532972+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"144'465","reported_seq":12527,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232213+0000","last_change":"2024-05-08T08:27:23.298683+0000","last_active":"2024-05-08T08:27:37.232213+0000","last_peered":"2024-05-08T08:27:37.232213+0000","last_clean":"2024-05-08T08:27:37.232213+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:37.232213+0000","last_undegraded":"2024-05-08T08:27:37.232213+0000","last_fullsized":"2024-05-08T08:27:37.232213+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:15.541483+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'433","reported_seq":2347,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226292+0000","last_change":"2024-05-08T08:27:13.121389+0000","last_active":"2024-05-08T08:27:36.226292+0000","last_peered":"2024-05-08T08:27:36.226292+0000","last_clean":"2024-05-08T08:27:36.226292+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:36.226292+0000","last_undegraded":"2024-05-08T08:27:36.226292+0000","last_fullsized":"2024-05-08T08:27:36.226292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:50.144904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'409","reported_seq":12780,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230842+0000","last_change":"2024-05-08T08:27:28.179669+0000","last_active":"2024-05-08T08:27:37.230842+0000","last_peered":"2024-05-08T08:27:37.230842+0000","last_clean":"2024-05-08T08:27:37.230842+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:37.230842+0000","last_undegraded":"2024-05-08T08:27:37.230842+0000","last_fullsized":"2024-05-08T08:27:37.230842+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:58.353989+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2479,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232346+0000","last_change":"2024-05-08T08:27:26.164009+0000","last_active":"2024-05-08T08:27:37.232346+0000","last_peered":"2024-05-08T08:27:37.232346+0000","last_clean":"2024-05-08T08:27:37.232346+0000","last_became_active":"2024-05-08T08:27:26.162218+0000","last_became_peered":"2024-05-08T08:27:26.162218+0000","last_unstale":"2024-05-08T08:27:37.232346+0000","last_undegraded":"2024-05-08T08:27:37.232346+0000","last_fullsized":"2024-05-08T08:27:37.232346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:24.166081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"136'390","reported_seq":2124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415661+0000","last_change":"2024-05-08T08:27:29.996790+0000","last_active":"2024-05-08T08:27:37.415661+0000","last_peered":"2024-05-08T08:27:37.415661+0000","last_clean":"2024-05-08T08:27:37.415661+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:37.415661+0000","last_undegraded":"2024-05-08T08:27:37.415661+0000","last_fullsized":"2024-05-08T08:27:37.415661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:44.557422+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"130'424","reported_seq":4167,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226484+0000","last_change":"2024-05-08T08:27:14.994770+0000","last_active":"2024-05-08T08:27:36.226484+0000","last_peered":"2024-05-08T08:27:36.226484+0000","last_clean":"2024-05-08T08:27:36.226484+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:36.226484+0000","last_undegraded":"2024-05-08T08:27:36.226484+0000","last_fullsized":"2024-05-08T08:27:36.226484+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:18:01.277964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"144'509","reported_seq":1509,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232287+0000","last_change":"2024-05-08T08:27:23.281639+0000","last_active":"2024-05-08T08:27:37.232287+0000","last_peered":"2024-05-08T08:27:37.232287+0000","last_clean":"2024-05-08T08:27:37.232287+0000","last_became_active":"2024-05-08T08:27:23.146211+0000","last_became_peered":"2024-05-08T08:27:23.146211+0000","last_unstale":"2024-05-08T08:27:37.232287+0000","last_undegraded":"2024-05-08T08:27:37.232287+0000","last_fullsized":"2024-05-08T08:27:37.232287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:58.260716+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413392+0000","last_change":"2024-05-08T08:27:05.057403+0000","last_active":"2024-05-08T08:27:36.413392+0000","last_peered":"2024-05-08T08:27:36.413392+0000","last_clean":"2024-05-08T08:27:36.413392+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:36.413392+0000","last_undegraded":"2024-05-08T08:27:36.413392+0000","last_fullsized":"2024-05-08T08:27:36.413392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:06.889029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"136'424","reported_seq":9197,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415706+0000","last_change":"2024-05-08T08:27:29.996807+0000","last_active":"2024-05-08T08:27:37.415706+0000","last_peered":"2024-05-08T08:27:37.415706+0000","last_clean":"2024-05-08T08:27:37.415706+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:37.415706+0000","last_undegraded":"2024-05-08T08:27:37.415706+0000","last_fullsized":"2024-05-08T08:27:37.415706+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:02.089980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413509+0000","last_change":"2024-05-08T08:27:05.061031+0000","last_active":"2024-05-08T08:27:36.413509+0000","last_peered":"2024-05-08T08:27:36.413509+0000","last_clean":"2024-05-08T08:27:36.413509+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:36.413509+0000","last_undegraded":"2024-05-08T08:27:36.413509+0000","last_fullsized":"2024-05-08T08:27:36.413509+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:49:14.437090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"136'452","reported_seq":5709,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:28.180565+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.778235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":235,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232386+0000","last_change":"2024-05-08T08:27:25.160288+0000","last_active":"2024-05-08T08:27:37.232386+0000","last_peered":"2024-05-08T08:27:37.232386+0000","last_clean":"2024-05-08T08:27:37.232386+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:37.232386+0000","last_undegraded":"2024-05-08T08:27:37.232386+0000","last_fullsized":"2024-05-08T08:27:37.232386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:49.667882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'850","reported_seq":7956,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231057+0000","last_change":"2024-05-08T08:27:28.182287+0000","last_active":"2024-05-08T08:27:37.231057+0000","last_peered":"2024-05-08T08:27:37.231057+0000","last_clean":"2024-05-08T08:27:37.231057+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:37.231057+0000","last_undegraded":"2024-05-08T08:27:37.231057+0000","last_fullsized":"2024-05-08T08:27:37.231057+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:45.798327+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":296,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231165+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:27:37.231165+0000","last_peered":"2024-05-08T08:27:37.231165+0000","last_clean":"2024-05-08T08:27:37.231165+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:37.231165+0000","last_undegraded":"2024-05-08T08:27:37.231165+0000","last_fullsized":"2024-05-08T08:27:37.231165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'513","reported_seq":3310,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054297+0000","last_change":"2024-05-08T08:27:13.130200+0000","last_active":"2024-05-08T08:27:38.054297+0000","last_peered":"2024-05-08T08:27:38.054297+0000","last_clean":"2024-05-08T08:27:38.054297+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:38.054297+0000","last_undegraded":"2024-05-08T08:27:38.054297+0000","last_fullsized":"2024-05-08T08:27:38.054297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:44.403890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"144'1264","reported_seq":3214,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054556+0000","last_change":"2024-05-08T08:27:23.316843+0000","last_active":"2024-05-08T08:27:38.054556+0000","last_peered":"2024-05-08T08:27:38.054556+0000","last_clean":"2024-05-08T08:27:38.054556+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:38.054556+0000","last_undegraded":"2024-05-08T08:27:38.054556+0000","last_fullsized":"2024-05-08T08:27:38.054556+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:47.097836+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":450,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416093+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:27:36.416093+0000","last_peered":"2024-05-08T08:27:36.416093+0000","last_clean":"2024-05-08T08:27:36.416093+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:36.416093+0000","last_undegraded":"2024-05-08T08:27:36.416093+0000","last_fullsized":"2024-05-08T08:27:36.416093+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":232,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:05.059497+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:50:56.853131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"136'363","reported_seq":1899,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226181+0000","last_change":"2024-05-08T08:27:13.120989+0000","last_active":"2024-05-08T08:27:36.226181+0000","last_peered":"2024-05-08T08:27:36.226181+0000","last_clean":"2024-05-08T08:27:36.226181+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:36.226181+0000","last_undegraded":"2024-05-08T08:27:36.226181+0000","last_fullsized":"2024-05-08T08:27:36.226181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:32.583458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416150+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:27:36.416150+0000","last_peered":"2024-05-08T08:27:36.416150+0000","last_clean":"2024-05-08T08:27:36.416150+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:36.416150+0000","last_undegraded":"2024-05-08T08:27:36.416150+0000","last_fullsized":"2024-05-08T08:27:36.416150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"137'797","reported_seq":6168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231399+0000","last_change":"2024-05-08T08:27:13.158703+0000","last_active":"2024-05-08T08:27:37.231399+0000","last_peered":"2024-05-08T08:27:37.231399+0000","last_clean":"2024-05-08T08:27:37.231399+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:37.231399+0000","last_undegraded":"2024-05-08T08:27:37.231399+0000","last_fullsized":"2024-05-08T08:27:37.231399+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:44:42.443755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415518+0000","last_change":"2024-05-08T08:27:29.992916+0000","last_active":"2024-05-08T08:27:37.415518+0000","last_peered":"2024-05-08T08:27:37.415518+0000","last_clean":"2024-05-08T08:27:37.415518+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:37.415518+0000","last_undegraded":"2024-05-08T08:27:37.415518+0000","last_fullsized":"2024-05-08T08:27:37.415518+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:54.323209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1976,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232087+0000","last_change":"2024-05-08T08:27:26.167343+0000","last_active":"2024-05-08T08:27:37.232087+0000","last_peered":"2024-05-08T08:27:37.232087+0000","last_clean":"2024-05-08T08:27:37.232087+0000","last_became_active":"2024-05-08T08:27:26.165179+0000","last_became_peered":"2024-05-08T08:27:26.165179+0000","last_unstale":"2024-05-08T08:27:37.232087+0000","last_undegraded":"2024-05-08T08:27:37.232087+0000","last_fullsized":"2024-05-08T08:27:37.232087+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:23.290092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416094+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:27:36.416094+0000","last_peered":"2024-05-08T08:27:36.416094+0000","last_clean":"2024-05-08T08:27:36.416094+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:36.416094+0000","last_undegraded":"2024-05-08T08:27:36.416094+0000","last_fullsized":"2024-05-08T08:27:36.416094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":4994,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232007+0000","last_change":"2024-05-08T08:27:26.163935+0000","last_active":"2024-05-08T08:27:37.232007+0000","last_peered":"2024-05-08T08:27:37.232007+0000","last_clean":"2024-05-08T08:27:37.232007+0000","last_became_active":"2024-05-08T08:27:26.161990+0000","last_became_peered":"2024-05-08T08:27:26.161990+0000","last_unstale":"2024-05-08T08:27:37.232007+0000","last_undegraded":"2024-05-08T08:27:37.232007+0000","last_fullsized":"2024-05-08T08:27:37.232007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:25.301862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"144'353","reported_seq":1591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232115+0000","last_change":"2024-05-08T08:27:23.249997+0000","last_active":"2024-05-08T08:27:37.232115+0000","last_peered":"2024-05-08T08:27:37.232115+0000","last_clean":"2024-05-08T08:27:37.232115+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:37.232115+0000","last_undegraded":"2024-05-08T08:27:37.232115+0000","last_fullsized":"2024-05-08T08:27:37.232115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:45:17.630472+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"144'668","reported_seq":1986,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230934+0000","last_change":"2024-05-08T08:27:28.183953+0000","last_active":"2024-05-08T08:27:37.230934+0000","last_peered":"2024-05-08T08:27:37.230934+0000","last_clean":"2024-05-08T08:27:37.230934+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:37.230934+0000","last_undegraded":"2024-05-08T08:27:37.230934+0000","last_fullsized":"2024-05-08T08:27:37.230934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:13:51.258724+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'368","reported_seq":2574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230915+0000","last_change":"2024-05-08T08:27:28.179656+0000","last_active":"2024-05-08T08:27:37.230915+0000","last_peered":"2024-05-08T08:27:37.230915+0000","last_clean":"2024-05-08T08:27:37.230915+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:37.230915+0000","last_undegraded":"2024-05-08T08:27:37.230915+0000","last_fullsized":"2024-05-08T08:27:37.230915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:06:30.701923+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"130'373","reported_seq":4568,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415547+0000","last_change":"2024-05-08T08:27:29.996890+0000","last_active":"2024-05-08T08:27:37.415547+0000","last_peered":"2024-05-08T08:27:37.415547+0000","last_clean":"2024-05-08T08:27:37.415547+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:37.415547+0000","last_undegraded":"2024-05-08T08:27:37.415547+0000","last_fullsized":"2024-05-08T08:27:37.415547+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.108598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"136'373","reported_seq":1881,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416251+0000","last_change":"2024-05-08T08:27:13.115549+0000","last_active":"2024-05-08T08:27:36.416251+0000","last_peered":"2024-05-08T08:27:36.416251+0000","last_clean":"2024-05-08T08:27:36.416251+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:36.416251+0000","last_undegraded":"2024-05-08T08:27:36.416251+0000","last_fullsized":"2024-05-08T08:27:36.416251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:23.190217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'660","reported_seq":5045,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416292+0000","last_change":"2024-05-08T08:27:24.170963+0000","last_active":"2024-05-08T08:27:36.416292+0000","last_peered":"2024-05-08T08:27:36.416292+0000","last_clean":"2024-05-08T08:27:36.416292+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:36.416292+0000","last_undegraded":"2024-05-08T08:27:36.416292+0000","last_fullsized":"2024-05-08T08:27:36.416292+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:30.030308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"144'457","reported_seq":1925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416300+0000","last_change":"2024-05-08T08:27:23.187752+0000","last_active":"2024-05-08T08:27:36.416300+0000","last_peered":"2024-05-08T08:27:36.416300+0000","last_clean":"2024-05-08T08:27:36.416300+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:36.416300+0000","last_undegraded":"2024-05-08T08:27:36.416300+0000","last_fullsized":"2024-05-08T08:27:36.416300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:05.388094+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"137'698","reported_seq":3067,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413387+0000","last_change":"2024-05-08T08:27:13.117579+0000","last_active":"2024-05-08T08:27:36.413387+0000","last_peered":"2024-05-08T08:27:36.413387+0000","last_clean":"2024-05-08T08:27:36.413387+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:36.413387+0000","last_undegraded":"2024-05-08T08:27:36.413387+0000","last_fullsized":"2024-05-08T08:27:36.413387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:05.872143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"144'464","reported_seq":1874,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416108+0000","last_change":"2024-05-08T08:27:23.343798+0000","last_active":"2024-05-08T08:27:36.416108+0000","last_peered":"2024-05-08T08:27:36.416108+0000","last_clean":"2024-05-08T08:27:36.416108+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:36.416108+0000","last_undegraded":"2024-05-08T08:27:36.416108+0000","last_fullsized":"2024-05-08T08:27:36.416108+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:34.081134+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"133'375","reported_seq":3462,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053982+0000","last_change":"2024-05-08T08:27:13.129448+0000","last_active":"2024-05-08T08:27:38.053982+0000","last_peered":"2024-05-08T08:27:38.053982+0000","last_clean":"2024-05-08T08:27:38.053982+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:38.053982+0000","last_undegraded":"2024-05-08T08:27:38.053982+0000","last_fullsized":"2024-05-08T08:27:38.053982+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:51:57.502154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"144'706","reported_seq":4459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416199+0000","last_change":"2024-05-08T08:27:23.275749+0000","last_active":"2024-05-08T08:27:36.416199+0000","last_peered":"2024-05-08T08:27:36.416199+0000","last_clean":"2024-05-08T08:27:36.416199+0000","last_became_active":"2024-05-08T08:27:23.151481+0000","last_became_peered":"2024-05-08T08:27:23.151481+0000","last_unstale":"2024-05-08T08:27:36.416199+0000","last_undegraded":"2024-05-08T08:27:36.416199+0000","last_fullsized":"2024-05-08T08:27:36.416199+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:45.998918+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18341,"ondisk_log_size":18341,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":1508748,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35412,"kb_used_data":1860,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336428,"statfs":{"total":96636764160,"available":96600502272,"internally_reserved":0,"allocated":1904640,"data_stored":924386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234009,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43068,"kb_used_data":1904,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328772,"statfs":{"total":96636764160,"available":96592662528,"internally_reserved":0,"allocated":1949696,"data_stored":1508767,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:27:42.135 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T08:27:42.135 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:27:42.433 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:42.433 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:27:42.445 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":234,"stamp":"2024-05-08T08:27:40.450726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18458,"ondisk_log_size":18458,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379420,"kb_used_data":14896,"kb_used_omap":55,"kb_used_meta":364360,"kb_avail":754595300,"statfs":{"total":773094113280,"available":772705587200,"internally_reserved":0,"allocated":15253504,"data_stored":8539788,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":373105167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001551"},"pg_stats":[{"pgid":"3.1f","version":"137'740","reported_seq":5498,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053843+0000","last_change":"2024-05-08T08:27:13.130208+0000","last_active":"2024-05-08T08:27:38.053843+0000","last_peered":"2024-05-08T08:27:38.053843+0000","last_clean":"2024-05-08T08:27:38.053843+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:38.053843+0000","last_undegraded":"2024-05-08T08:27:38.053843+0000","last_fullsized":"2024-05-08T08:27:38.053843+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:12:52.764201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'796","reported_seq":3762,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230931+0000","last_change":"2024-05-08T08:27:28.179242+0000","last_active":"2024-05-08T08:27:37.230931+0000","last_peered":"2024-05-08T08:27:37.230931+0000","last_clean":"2024-05-08T08:27:37.230931+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:37.230931+0000","last_undegraded":"2024-05-08T08:27:37.230931+0000","last_fullsized":"2024-05-08T08:27:37.230931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:09.650651+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"144'562","reported_seq":2136,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415591+0000","last_change":"2024-05-08T08:27:30.000096+0000","last_active":"2024-05-08T08:27:37.415591+0000","last_peered":"2024-05-08T08:27:37.415591+0000","last_clean":"2024-05-08T08:27:37.415591+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:37.415591+0000","last_undegraded":"2024-05-08T08:27:37.415591+0000","last_fullsized":"2024-05-08T08:27:37.415591+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:17:13.519824+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"144'794","reported_seq":7674,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415540+0000","last_change":"2024-05-08T08:27:30.000107+0000","last_active":"2024-05-08T08:27:37.415540+0000","last_peered":"2024-05-08T08:27:37.415540+0000","last_clean":"2024-05-08T08:27:37.415540+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:37.415540+0000","last_undegraded":"2024-05-08T08:27:37.415540+0000","last_fullsized":"2024-05-08T08:27:37.415540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:45.406288+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"144'1100","reported_seq":2794,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054757+0000","last_change":"2024-05-08T08:27:23.244589+0000","last_active":"2024-05-08T08:27:38.054757+0000","last_peered":"2024-05-08T08:27:38.054757+0000","last_clean":"2024-05-08T08:27:38.054757+0000","last_became_active":"2024-05-08T08:27:23.151929+0000","last_became_peered":"2024-05-08T08:27:23.151929+0000","last_unstale":"2024-05-08T08:27:38.054757+0000","last_undegraded":"2024-05-08T08:27:38.054757+0000","last_fullsized":"2024-05-08T08:27:38.054757+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:07:18.532972+0000","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"144'465","reported_seq":12527,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232213+0000","last_change":"2024-05-08T08:27:23.298683+0000","last_active":"2024-05-08T08:27:37.232213+0000","last_peered":"2024-05-08T08:27:37.232213+0000","last_clean":"2024-05-08T08:27:37.232213+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:37.232213+0000","last_undegraded":"2024-05-08T08:27:37.232213+0000","last_fullsized":"2024-05-08T08:27:37.232213+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:15.541483+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'433","reported_seq":2347,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226292+0000","last_change":"2024-05-08T08:27:13.121389+0000","last_active":"2024-05-08T08:27:36.226292+0000","last_peered":"2024-05-08T08:27:36.226292+0000","last_clean":"2024-05-08T08:27:36.226292+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:36.226292+0000","last_undegraded":"2024-05-08T08:27:36.226292+0000","last_fullsized":"2024-05-08T08:27:36.226292+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:50.144904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'409","reported_seq":12780,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230842+0000","last_change":"2024-05-08T08:27:28.179669+0000","last_active":"2024-05-08T08:27:37.230842+0000","last_peered":"2024-05-08T08:27:37.230842+0000","last_clean":"2024-05-08T08:27:37.230842+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:37.230842+0000","last_undegraded":"2024-05-08T08:27:37.230842+0000","last_fullsized":"2024-05-08T08:27:37.230842+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:58.353989+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2479,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232346+0000","last_change":"2024-05-08T08:27:26.164009+0000","last_active":"2024-05-08T08:27:37.232346+0000","last_peered":"2024-05-08T08:27:37.232346+0000","last_clean":"2024-05-08T08:27:37.232346+0000","last_became_active":"2024-05-08T08:27:26.162218+0000","last_became_peered":"2024-05-08T08:27:26.162218+0000","last_unstale":"2024-05-08T08:27:37.232346+0000","last_undegraded":"2024-05-08T08:27:37.232346+0000","last_fullsized":"2024-05-08T08:27:37.232346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:24.166081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"136'390","reported_seq":2124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415661+0000","last_change":"2024-05-08T08:27:29.996790+0000","last_active":"2024-05-08T08:27:37.415661+0000","last_peered":"2024-05-08T08:27:37.415661+0000","last_clean":"2024-05-08T08:27:37.415661+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:37.415661+0000","last_undegraded":"2024-05-08T08:27:37.415661+0000","last_fullsized":"2024-05-08T08:27:37.415661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:44.557422+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"130'424","reported_seq":4167,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226484+0000","last_change":"2024-05-08T08:27:14.994770+0000","last_active":"2024-05-08T08:27:36.226484+0000","last_peered":"2024-05-08T08:27:36.226484+0000","last_clean":"2024-05-08T08:27:36.226484+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:36.226484+0000","last_undegraded":"2024-05-08T08:27:36.226484+0000","last_fullsized":"2024-05-08T08:27:36.226484+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:18:01.277964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"144'509","reported_seq":1509,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232287+0000","last_change":"2024-05-08T08:27:23.281639+0000","last_active":"2024-05-08T08:27:37.232287+0000","last_peered":"2024-05-08T08:27:37.232287+0000","last_clean":"2024-05-08T08:27:37.232287+0000","last_became_active":"2024-05-08T08:27:23.146211+0000","last_became_peered":"2024-05-08T08:27:23.146211+0000","last_unstale":"2024-05-08T08:27:37.232287+0000","last_undegraded":"2024-05-08T08:27:37.232287+0000","last_fullsized":"2024-05-08T08:27:37.232287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:58.260716+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413392+0000","last_change":"2024-05-08T08:27:05.057403+0000","last_active":"2024-05-08T08:27:36.413392+0000","last_peered":"2024-05-08T08:27:36.413392+0000","last_clean":"2024-05-08T08:27:36.413392+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:36.413392+0000","last_undegraded":"2024-05-08T08:27:36.413392+0000","last_fullsized":"2024-05-08T08:27:36.413392+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:06.889029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"136'424","reported_seq":9197,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415706+0000","last_change":"2024-05-08T08:27:29.996807+0000","last_active":"2024-05-08T08:27:37.415706+0000","last_peered":"2024-05-08T08:27:37.415706+0000","last_clean":"2024-05-08T08:27:37.415706+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:37.415706+0000","last_undegraded":"2024-05-08T08:27:37.415706+0000","last_fullsized":"2024-05-08T08:27:37.415706+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:02.089980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":66,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413509+0000","last_change":"2024-05-08T08:27:05.061031+0000","last_active":"2024-05-08T08:27:36.413509+0000","last_peered":"2024-05-08T08:27:36.413509+0000","last_clean":"2024-05-08T08:27:36.413509+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:36.413509+0000","last_undegraded":"2024-05-08T08:27:36.413509+0000","last_fullsized":"2024-05-08T08:27:36.413509+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:49:14.437090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"136'452","reported_seq":5709,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:28.180565+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.778235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":235,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232386+0000","last_change":"2024-05-08T08:27:25.160288+0000","last_active":"2024-05-08T08:27:37.232386+0000","last_peered":"2024-05-08T08:27:37.232386+0000","last_clean":"2024-05-08T08:27:37.232386+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:37.232386+0000","last_undegraded":"2024-05-08T08:27:37.232386+0000","last_fullsized":"2024-05-08T08:27:37.232386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:49.667882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'850","reported_seq":7956,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231057+0000","last_change":"2024-05-08T08:27:28.182287+0000","last_active":"2024-05-08T08:27:37.231057+0000","last_peered":"2024-05-08T08:27:37.231057+0000","last_clean":"2024-05-08T08:27:37.231057+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:37.231057+0000","last_undegraded":"2024-05-08T08:27:37.231057+0000","last_fullsized":"2024-05-08T08:27:37.231057+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:45.798327+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":296,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231165+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:27:37.231165+0000","last_peered":"2024-05-08T08:27:37.231165+0000","last_clean":"2024-05-08T08:27:37.231165+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:37.231165+0000","last_undegraded":"2024-05-08T08:27:37.231165+0000","last_fullsized":"2024-05-08T08:27:37.231165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"137'513","reported_seq":3310,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054297+0000","last_change":"2024-05-08T08:27:13.130200+0000","last_active":"2024-05-08T08:27:38.054297+0000","last_peered":"2024-05-08T08:27:38.054297+0000","last_clean":"2024-05-08T08:27:38.054297+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:38.054297+0000","last_undegraded":"2024-05-08T08:27:38.054297+0000","last_fullsized":"2024-05-08T08:27:38.054297+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":513,"log_dups_size":0,"ondisk_log_size":513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:01:44.403890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"144'1264","reported_seq":3214,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054556+0000","last_change":"2024-05-08T08:27:23.316843+0000","last_active":"2024-05-08T08:27:38.054556+0000","last_peered":"2024-05-08T08:27:38.054556+0000","last_clean":"2024-05-08T08:27:38.054556+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:38.054556+0000","last_undegraded":"2024-05-08T08:27:38.054556+0000","last_fullsized":"2024-05-08T08:27:38.054556+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:29:47.097836+0000","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":450,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416093+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:27:36.416093+0000","last_peered":"2024-05-08T08:27:36.416093+0000","last_clean":"2024-05-08T08:27:36.416093+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:36.416093+0000","last_undegraded":"2024-05-08T08:27:36.416093+0000","last_fullsized":"2024-05-08T08:27:36.416093+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":232,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226480+0000","last_change":"2024-05-08T08:27:05.059497+0000","last_active":"2024-05-08T08:27:36.226480+0000","last_peered":"2024-05-08T08:27:36.226480+0000","last_clean":"2024-05-08T08:27:36.226480+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:36.226480+0000","last_undegraded":"2024-05-08T08:27:36.226480+0000","last_fullsized":"2024-05-08T08:27:36.226480+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:50:56.853131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"136'363","reported_seq":1899,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.226181+0000","last_change":"2024-05-08T08:27:13.120989+0000","last_active":"2024-05-08T08:27:36.226181+0000","last_peered":"2024-05-08T08:27:36.226181+0000","last_clean":"2024-05-08T08:27:36.226181+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:36.226181+0000","last_undegraded":"2024-05-08T08:27:36.226181+0000","last_fullsized":"2024-05-08T08:27:36.226181+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:32.583458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416150+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:27:36.416150+0000","last_peered":"2024-05-08T08:27:36.416150+0000","last_clean":"2024-05-08T08:27:36.416150+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:36.416150+0000","last_undegraded":"2024-05-08T08:27:36.416150+0000","last_fullsized":"2024-05-08T08:27:36.416150+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"137'797","reported_seq":6168,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231399+0000","last_change":"2024-05-08T08:27:13.158703+0000","last_active":"2024-05-08T08:27:37.231399+0000","last_peered":"2024-05-08T08:27:37.231399+0000","last_clean":"2024-05-08T08:27:37.231399+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:37.231399+0000","last_undegraded":"2024-05-08T08:27:37.231399+0000","last_fullsized":"2024-05-08T08:27:37.231399+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:44:42.443755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415518+0000","last_change":"2024-05-08T08:27:29.992916+0000","last_active":"2024-05-08T08:27:37.415518+0000","last_peered":"2024-05-08T08:27:37.415518+0000","last_clean":"2024-05-08T08:27:37.415518+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:37.415518+0000","last_undegraded":"2024-05-08T08:27:37.415518+0000","last_fullsized":"2024-05-08T08:27:37.415518+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:54.323209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1976,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232087+0000","last_change":"2024-05-08T08:27:26.167343+0000","last_active":"2024-05-08T08:27:37.232087+0000","last_peered":"2024-05-08T08:27:37.232087+0000","last_clean":"2024-05-08T08:27:37.232087+0000","last_became_active":"2024-05-08T08:27:26.165179+0000","last_became_peered":"2024-05-08T08:27:26.165179+0000","last_unstale":"2024-05-08T08:27:37.232087+0000","last_undegraded":"2024-05-08T08:27:37.232087+0000","last_fullsized":"2024-05-08T08:27:37.232087+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:23.290092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":295,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416094+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:27:36.416094+0000","last_peered":"2024-05-08T08:27:36.416094+0000","last_clean":"2024-05-08T08:27:36.416094+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:36.416094+0000","last_undegraded":"2024-05-08T08:27:36.416094+0000","last_fullsized":"2024-05-08T08:27:36.416094+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":4994,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232007+0000","last_change":"2024-05-08T08:27:26.163935+0000","last_active":"2024-05-08T08:27:37.232007+0000","last_peered":"2024-05-08T08:27:37.232007+0000","last_clean":"2024-05-08T08:27:37.232007+0000","last_became_active":"2024-05-08T08:27:26.161990+0000","last_became_peered":"2024-05-08T08:27:26.161990+0000","last_unstale":"2024-05-08T08:27:37.232007+0000","last_undegraded":"2024-05-08T08:27:37.232007+0000","last_fullsized":"2024-05-08T08:27:37.232007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:25.301862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"144'353","reported_seq":1591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232115+0000","last_change":"2024-05-08T08:27:23.249997+0000","last_active":"2024-05-08T08:27:37.232115+0000","last_peered":"2024-05-08T08:27:37.232115+0000","last_clean":"2024-05-08T08:27:37.232115+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:37.232115+0000","last_undegraded":"2024-05-08T08:27:37.232115+0000","last_fullsized":"2024-05-08T08:27:37.232115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:45:17.630472+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"144'668","reported_seq":1986,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230934+0000","last_change":"2024-05-08T08:27:28.183953+0000","last_active":"2024-05-08T08:27:37.230934+0000","last_peered":"2024-05-08T08:27:37.230934+0000","last_clean":"2024-05-08T08:27:37.230934+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:37.230934+0000","last_undegraded":"2024-05-08T08:27:37.230934+0000","last_fullsized":"2024-05-08T08:27:37.230934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:13:51.258724+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'368","reported_seq":2574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230915+0000","last_change":"2024-05-08T08:27:28.179656+0000","last_active":"2024-05-08T08:27:37.230915+0000","last_peered":"2024-05-08T08:27:37.230915+0000","last_clean":"2024-05-08T08:27:37.230915+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:37.230915+0000","last_undegraded":"2024-05-08T08:27:37.230915+0000","last_fullsized":"2024-05-08T08:27:37.230915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:06:30.701923+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"130'373","reported_seq":4568,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415547+0000","last_change":"2024-05-08T08:27:29.996890+0000","last_active":"2024-05-08T08:27:37.415547+0000","last_peered":"2024-05-08T08:27:37.415547+0000","last_clean":"2024-05-08T08:27:37.415547+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:37.415547+0000","last_undegraded":"2024-05-08T08:27:37.415547+0000","last_fullsized":"2024-05-08T08:27:37.415547+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.108598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"136'373","reported_seq":1881,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416251+0000","last_change":"2024-05-08T08:27:13.115549+0000","last_active":"2024-05-08T08:27:36.416251+0000","last_peered":"2024-05-08T08:27:36.416251+0000","last_clean":"2024-05-08T08:27:36.416251+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:36.416251+0000","last_undegraded":"2024-05-08T08:27:36.416251+0000","last_fullsized":"2024-05-08T08:27:36.416251+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:23.190217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'660","reported_seq":5045,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416292+0000","last_change":"2024-05-08T08:27:24.170963+0000","last_active":"2024-05-08T08:27:36.416292+0000","last_peered":"2024-05-08T08:27:36.416292+0000","last_clean":"2024-05-08T08:27:36.416292+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:36.416292+0000","last_undegraded":"2024-05-08T08:27:36.416292+0000","last_fullsized":"2024-05-08T08:27:36.416292+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:30.030308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"144'457","reported_seq":1925,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416300+0000","last_change":"2024-05-08T08:27:23.187752+0000","last_active":"2024-05-08T08:27:36.416300+0000","last_peered":"2024-05-08T08:27:36.416300+0000","last_clean":"2024-05-08T08:27:36.416300+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:36.416300+0000","last_undegraded":"2024-05-08T08:27:36.416300+0000","last_fullsized":"2024-05-08T08:27:36.416300+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:05.388094+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"137'698","reported_seq":3067,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.413387+0000","last_change":"2024-05-08T08:27:13.117579+0000","last_active":"2024-05-08T08:27:36.413387+0000","last_peered":"2024-05-08T08:27:36.413387+0000","last_clean":"2024-05-08T08:27:36.413387+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:36.413387+0000","last_undegraded":"2024-05-08T08:27:36.413387+0000","last_fullsized":"2024-05-08T08:27:36.413387+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:05.872143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"144'464","reported_seq":1874,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416108+0000","last_change":"2024-05-08T08:27:23.343798+0000","last_active":"2024-05-08T08:27:36.416108+0000","last_peered":"2024-05-08T08:27:36.416108+0000","last_clean":"2024-05-08T08:27:36.416108+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:36.416108+0000","last_undegraded":"2024-05-08T08:27:36.416108+0000","last_fullsized":"2024-05-08T08:27:36.416108+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:34.081134+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"133'375","reported_seq":3462,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053982+0000","last_change":"2024-05-08T08:27:13.129448+0000","last_active":"2024-05-08T08:27:38.053982+0000","last_peered":"2024-05-08T08:27:38.053982+0000","last_clean":"2024-05-08T08:27:38.053982+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:38.053982+0000","last_undegraded":"2024-05-08T08:27:38.053982+0000","last_fullsized":"2024-05-08T08:27:38.053982+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:51:57.502154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"144'706","reported_seq":4459,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-05-08T08:27:36.416199+0000","last_change":"2024-05-08T08:27:23.275749+0000","last_active":"2024-05-08T08:27:36.416199+0000","last_peered":"2024-05-08T08:27:36.416199+0000","last_clean":"2024-05-08T08:27:36.416199+0000","last_became_active":"2024-05-08T08:27:23.151481+0000","last_became_peered":"2024-05-08T08:27:23.151481+0000","last_unstale":"2024-05-08T08:27:36.416199+0000","last_undegraded":"2024-05-08T08:27:36.416199+0000","last_fullsized":"2024-05-08T08:27:36.416199+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:45.998918+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18341,"ondisk_log_size":18341,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51128,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320712,"statfs":{"total":96636764160,"available":96584409088,"internally_reserved":0,"allocated":1945600,"data_stored":1508748,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28864,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342976,"statfs":{"total":96636764160,"available":96607207424,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55232,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316608,"statfs":{"total":96636764160,"available":96580206592,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323333,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35412,"kb_used_data":1860,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336428,"statfs":{"total":96636764160,"available":96600502272,"internally_reserved":0,"allocated":1904640,"data_stored":924386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574885,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51092,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320748,"statfs":{"total":96636764160,"available":96584445952,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574884,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37184,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334656,"statfs":{"total":96636764160,"available":96598687744,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234009,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43068,"kb_used_data":1904,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328772,"statfs":{"total":96636764160,"available":96592662528,"internally_reserved":0,"allocated":1949696,"data_stored":1508767,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77440,"kb_used_data":1844,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294400,"statfs":{"total":96636764160,"available":96557465600,"internally_reserved":0,"allocated":1888256,"data_stored":918380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:27:42.447 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T08:27:42.747 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:42.747 INFO:teuthology.orchestra.run.smithi031.stdout:{"epoch":158,"fsid":"1b938f15-f9fa-4bd9-bd05-a0b67ea75bbf","created":"2024-05-08T08:24:44.143941+0000","modified":"2024-05-08T08:27:37.227345+0000","last_up_change":"2024-05-08T08:27:22.131410+0000","last_in_change":"2024-05-08T08:27:04.049694+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-05-08T08:24:55.134472+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2024-05-08T08:24:56.465515+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-05-08T08:25:03.054989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":137,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"14afea42-356a-4163-ac3f-2e270e807917","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6801","nonce":727789107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6803","nonce":727789107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6807","nonce":727789107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":727789107},{"type":"v1","addr":"172.21.15.31:6805","nonce":727789107}]},"public_addr":"172.21.15.31:6801/727789107","cluster_addr":"172.21.15.31:6803/727789107","heartbeat_back_addr":"172.21.15.31:6807/727789107","heartbeat_front_addr":"172.21.15.31:6805/727789107","state":["exists","up"]},{"osd":1,"uuid":"db77a923-8f49-4b47-b752-00b475917a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":30,"up_from":54,"up_thru":148,"down_at":32,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6817","nonce":2489759853}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6819","nonce":2489759853}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6823","nonce":2489759853}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":2489759853},{"type":"v1","addr":"172.21.15.31:6821","nonce":2489759853}]},"public_addr":"172.21.15.31:6817/2489759853","cluster_addr":"172.21.15.31:6819/2489759853","heartbeat_back_addr":"172.21.15.31:6823/2489759853","heartbeat_front_addr":"172.21.15.31:6821/2489759853","state":["exists","up"]},{"osd":2,"uuid":"3d508691-8e40-4592-82d0-eaf4c4c7d7dd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":110,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6809","nonce":2736372115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6811","nonce":2736372115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6815","nonce":2736372115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":2736372115},{"type":"v1","addr":"172.21.15.31:6813","nonce":2736372115}]},"public_addr":"172.21.15.31:6809/2736372115","cluster_addr":"172.21.15.31:6811/2736372115","heartbeat_back_addr":"172.21.15.31:6815/2736372115","heartbeat_front_addr":"172.21.15.31:6813/2736372115","state":["exists","up"]},{"osd":3,"uuid":"2545300a-428a-41ed-b19a-3a0cb9e5e755","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":148,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6825","nonce":573482428}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6827","nonce":573482428}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6831","nonce":573482428}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":573482428},{"type":"v1","addr":"172.21.15.31:6829","nonce":573482428}]},"public_addr":"172.21.15.31:6825/573482428","cluster_addr":"172.21.15.31:6827/573482428","heartbeat_back_addr":"172.21.15.31:6831/573482428","heartbeat_front_addr":"172.21.15.31:6829/573482428","state":["exists","up"]},{"osd":4,"uuid":"886aeed5-4495-4064-8579-050d38a4ad1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":108,"up_from":143,"up_thru":146,"down_at":110,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6801","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6803","nonce":2548527301}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6805","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6807","nonce":2548527301}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6813","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6815","nonce":2548527301}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6809","nonce":2548527301},{"type":"v1","addr":"172.21.15.149:6811","nonce":2548527301}]},"public_addr":"172.21.15.149:6803/2548527301","cluster_addr":"172.21.15.149:6807/2548527301","heartbeat_back_addr":"172.21.15.149:6815/2548527301","heartbeat_front_addr":"172.21.15.149:6811/2548527301","state":["exists","up"]},{"osd":5,"uuid":"4d15cef0-9ee9-4069-a6c4-a650e873ee03","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6816","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6818","nonce":3709461555}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6820","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6822","nonce":3709461555}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6828","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6830","nonce":3709461555}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6824","nonce":3709461555},{"type":"v1","addr":"172.21.15.149:6826","nonce":3709461555}]},"public_addr":"172.21.15.149:6818/3709461555","cluster_addr":"172.21.15.149:6822/3709461555","heartbeat_back_addr":"172.21.15.149:6830/3709461555","heartbeat_front_addr":"172.21.15.149:6826/3709461555","state":["exists","up"]},{"osd":6,"uuid":"910ac664-9573-4851-9e2b-89ac6015fdc1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6800","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6802","nonce":405561966}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6804","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6806","nonce":405561966}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6812","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6814","nonce":405561966}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6808","nonce":405561966},{"type":"v1","addr":"172.21.15.149:6810","nonce":405561966}]},"public_addr":"172.21.15.149:6802/405561966","cluster_addr":"172.21.15.149:6806/405561966","heartbeat_back_addr":"172.21.15.149:6814/405561966","heartbeat_front_addr":"172.21.15.149:6810/405561966","state":["exists","up"]},{"osd":7,"uuid":"14cc2d94-48e4-47c9-896a-9be9281ef0c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":144,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6817","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6819","nonce":570236217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6821","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6823","nonce":570236217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6829","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6831","nonce":570236217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.149:6825","nonce":570236217},{"type":"v1","addr":"172.21.15.149:6827","nonce":570236217}]},"public_addr":"172.21.15.149:6819/570236217","cluster_addr":"172.21.15.149:6823/570236217","heartbeat_back_addr":"172.21.15.149:6831/570236217","heartbeat_front_addr":"172.21.15.149:6827/570236217","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.572981+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.626543+0000","dead_epoch":32},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.638878+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.599293+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.565676+0000","dead_epoch":110},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.552124+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.531428+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T08:24:51.658391+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"3.3","osds":[2,1]},{"pgid":"3.a","osds":[3,4]},{"pgid":"3.15","osds":[1,7]}],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":4}]},{"pgid":"3.8","mappings":[{"from":3,"to":4}]},{"pgid":"3.17","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T08:27:43.761 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T08:27:43.761 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:44.084 INFO:teuthology.orchestra.run.smithi031.stderr:2024-05-08T08:27:44.092+0000 7f7607fff640 0 --2- 172.21.15.31:0/3512391076 >> v2:172.21.15.31:3300/0 conn(0x7f76080af880 0x7f760814a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-08T08:27:44.106 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:44.107 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:27:44.107 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:44.119 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T08:27:44.424 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T08:27:44.437 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T08:27:44.437 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:44.567 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:27:44.567 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:44.578 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T08:27:44.876 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T08:27:44.889 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T08:27:44.890 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:45.018 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:45.018 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:27:45.018 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:45.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T08:27:45.332 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T08:27:45.345 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T08:27:45.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:45.474 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:45.474 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:27:45.474 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:45.485 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T08:27:45.784 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T08:27:45.797 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T08:27:45.797 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:45.927 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:45.928 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2024-05-08T08:27:45.928 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:45.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T08:27:46.243 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T08:27:46.257 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T08:27:46.257 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:46.386 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:46.386 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:27:46.386 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:46.397 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T08:27:46.698 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T08:27:46.711 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T08:27:46.711 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:46.841 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:46.841 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:27:46.841 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:46.852 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T08:27:47.154 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T08:27:47.167 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T08:27:47.167 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T08:27:47.297 INFO:teuthology.orchestra.run.smithi031.stdout:{ 2024-05-08T08:27:47.297 INFO:teuthology.orchestra.run.smithi031.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T08:27:47.297 INFO:teuthology.orchestra.run.smithi031.stdout:} 2024-05-08T08:27:47.308 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T08:27:47.607 INFO:teuthology.orchestra.run.smithi031.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T08:27:47.620 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:27:47.919 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:27:47.920 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:27:47.931 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-05-08T08:27:46.452032+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18470,"ondisk_log_size":18470,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":379504,"kb_used_data":14968,"kb_used_omap":55,"kb_used_meta":364360,"kb_avail":754595216,"statfs":{"total":773094113280,"available":772705501184,"internally_reserved":0,"allocated":15327232,"data_stored":8575710,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":373105167},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002468"},"pg_stats":[{"pgid":"3.1f","version":"137'740","reported_seq":5499,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.053906+0000","last_change":"2024-05-08T08:27:13.130208+0000","last_active":"2024-05-08T08:27:38.053906+0000","last_peered":"2024-05-08T08:27:38.053906+0000","last_clean":"2024-05-08T08:27:38.053906+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:38.053906+0000","last_undegraded":"2024-05-08T08:27:38.053906+0000","last_fullsized":"2024-05-08T08:27:38.053906+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":740,"log_dups_size":0,"ondisk_log_size":740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"136'796","reported_seq":3762,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230931+0000","last_change":"2024-05-08T08:27:28.179242+0000","last_active":"2024-05-08T08:27:37.230931+0000","last_peered":"2024-05-08T08:27:37.230931+0000","last_clean":"2024-05-08T08:27:37.230931+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:37.230931+0000","last_undegraded":"2024-05-08T08:27:37.230931+0000","last_fullsized":"2024-05-08T08:27:37.230931+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":796,"log_dups_size":0,"ondisk_log_size":796,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:41:09.650651+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"144'562","reported_seq":2136,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415591+0000","last_change":"2024-05-08T08:27:30.000096+0000","last_active":"2024-05-08T08:27:37.415591+0000","last_peered":"2024-05-08T08:27:37.415591+0000","last_clean":"2024-05-08T08:27:37.415591+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:37.415591+0000","last_undegraded":"2024-05-08T08:27:37.415591+0000","last_fullsized":"2024-05-08T08:27:37.415591+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":562,"log_dups_size":0,"ondisk_log_size":562,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:17:13.519824+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"144'794","reported_seq":7674,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415540+0000","last_change":"2024-05-08T08:27:30.000107+0000","last_active":"2024-05-08T08:27:37.415540+0000","last_peered":"2024-05-08T08:27:37.415540+0000","last_clean":"2024-05-08T08:27:37.415540+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:37.415540+0000","last_undegraded":"2024-05-08T08:27:37.415540+0000","last_fullsized":"2024-05-08T08:27:37.415540+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":794,"log_dups_size":0,"ondisk_log_size":794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:20:45.406288+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"144'1100","reported_seq":2795,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054797+0000","last_change":"2024-05-08T08:27:23.244589+0000","last_active":"2024-05-08T08:27:38.054797+0000","last_peered":"2024-05-08T08:27:38.054797+0000","last_clean":"2024-05-08T08:27:38.054797+0000","last_became_active":"2024-05-08T08:27:23.151929+0000","last_became_peered":"2024-05-08T08:27:23.151929+0000","last_unstale":"2024-05-08T08:27:38.054797+0000","last_undegraded":"2024-05-08T08:27:38.054797+0000","last_fullsized":"2024-05-08T08:27:38.054797+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1100,"log_dups_size":0,"ondisk_log_size":1100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"144'465","reported_seq":12527,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232213+0000","last_change":"2024-05-08T08:27:23.298683+0000","last_active":"2024-05-08T08:27:37.232213+0000","last_peered":"2024-05-08T08:27:37.232213+0000","last_clean":"2024-05-08T08:27:37.232213+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:37.232213+0000","last_undegraded":"2024-05-08T08:27:37.232213+0000","last_fullsized":"2024-05-08T08:27:37.232213+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:20:15.541483+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"136'433","reported_seq":2349,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.511996+0000","last_change":"2024-05-08T08:27:13.121389+0000","last_active":"2024-05-08T08:27:38.511996+0000","last_peered":"2024-05-08T08:27:38.511996+0000","last_clean":"2024-05-08T08:27:38.511996+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:38.511996+0000","last_undegraded":"2024-05-08T08:27:38.511996+0000","last_fullsized":"2024-05-08T08:27:38.511996+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:32:50.144904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"136'409","reported_seq":12780,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230842+0000","last_change":"2024-05-08T08:27:28.179669+0000","last_active":"2024-05-08T08:27:37.230842+0000","last_peered":"2024-05-08T08:27:37.230842+0000","last_clean":"2024-05-08T08:27:37.230842+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:37.230842+0000","last_undegraded":"2024-05-08T08:27:37.230842+0000","last_fullsized":"2024-05-08T08:27:37.230842+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":409,"log_dups_size":0,"ondisk_log_size":409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:58.353989+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2479,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232346+0000","last_change":"2024-05-08T08:27:26.164009+0000","last_active":"2024-05-08T08:27:37.232346+0000","last_peered":"2024-05-08T08:27:37.232346+0000","last_clean":"2024-05-08T08:27:37.232346+0000","last_became_active":"2024-05-08T08:27:26.162218+0000","last_became_peered":"2024-05-08T08:27:26.162218+0000","last_unstale":"2024-05-08T08:27:37.232346+0000","last_undegraded":"2024-05-08T08:27:37.232346+0000","last_fullsized":"2024-05-08T08:27:37.232346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:47:24.166081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":303,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.16","version":"136'390","reported_seq":2124,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415661+0000","last_change":"2024-05-08T08:27:29.996790+0000","last_active":"2024-05-08T08:27:37.415661+0000","last_peered":"2024-05-08T08:27:37.415661+0000","last_clean":"2024-05-08T08:27:37.415661+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:37.415661+0000","last_undegraded":"2024-05-08T08:27:37.415661+0000","last_fullsized":"2024-05-08T08:27:37.415661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:04:44.557422+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"130'424","reported_seq":4169,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.511792+0000","last_change":"2024-05-08T08:27:14.994770+0000","last_active":"2024-05-08T08:27:38.511792+0000","last_peered":"2024-05-08T08:27:38.511792+0000","last_clean":"2024-05-08T08:27:38.511792+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:38.511792+0000","last_undegraded":"2024-05-08T08:27:38.511792+0000","last_fullsized":"2024-05-08T08:27:38.511792+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:18:01.277964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"144'509","reported_seq":1509,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232287+0000","last_change":"2024-05-08T08:27:23.281639+0000","last_active":"2024-05-08T08:27:37.232287+0000","last_peered":"2024-05-08T08:27:37.232287+0000","last_clean":"2024-05-08T08:27:37.232287+0000","last_became_active":"2024-05-08T08:27:23.146211+0000","last_became_peered":"2024-05-08T08:27:23.146211+0000","last_unstale":"2024-05-08T08:27:37.232287+0000","last_undegraded":"2024-05-08T08:27:37.232287+0000","last_fullsized":"2024-05-08T08:27:37.232287+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":509,"log_dups_size":0,"ondisk_log_size":509,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:05:58.260716+0000","scrub_duration":0,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":68,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.511684+0000","last_change":"2024-05-08T08:27:05.057403+0000","last_active":"2024-05-08T08:27:38.511684+0000","last_peered":"2024-05-08T08:27:38.511684+0000","last_clean":"2024-05-08T08:27:38.511684+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:38.511684+0000","last_undegraded":"2024-05-08T08:27:38.511684+0000","last_fullsized":"2024-05-08T08:27:38.511684+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:47:06.889029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"136'424","reported_seq":9197,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415706+0000","last_change":"2024-05-08T08:27:29.996807+0000","last_active":"2024-05-08T08:27:37.415706+0000","last_peered":"2024-05-08T08:27:37.415706+0000","last_clean":"2024-05-08T08:27:37.415706+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:37.415706+0000","last_undegraded":"2024-05-08T08:27:37.415706+0000","last_fullsized":"2024-05-08T08:27:37.415706+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":424,"log_dups_size":0,"ondisk_log_size":424,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:02.089980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":68,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.511658+0000","last_change":"2024-05-08T08:27:05.061031+0000","last_active":"2024-05-08T08:27:38.511658+0000","last_peered":"2024-05-08T08:27:38.511658+0000","last_clean":"2024-05-08T08:27:38.511658+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:38.511658+0000","last_undegraded":"2024-05-08T08:27:38.511658+0000","last_fullsized":"2024-05-08T08:27:38.511658+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:49:14.437090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"136'452","reported_seq":5711,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.512514+0000","last_change":"2024-05-08T08:27:28.180565+0000","last_active":"2024-05-08T08:27:38.512514+0000","last_peered":"2024-05-08T08:27:38.512514+0000","last_clean":"2024-05-08T08:27:38.512514+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:38.512514+0000","last_undegraded":"2024-05-08T08:27:38.512514+0000","last_fullsized":"2024-05-08T08:27:38.512514+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:13:24.778235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":235,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232386+0000","last_change":"2024-05-08T08:27:25.160288+0000","last_active":"2024-05-08T08:27:37.232386+0000","last_peered":"2024-05-08T08:27:37.232386+0000","last_clean":"2024-05-08T08:27:37.232386+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:37.232386+0000","last_undegraded":"2024-05-08T08:27:37.232386+0000","last_fullsized":"2024-05-08T08:27:37.232386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:49.667882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"136'850","reported_seq":7956,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231057+0000","last_change":"2024-05-08T08:27:28.182287+0000","last_active":"2024-05-08T08:27:37.231057+0000","last_peered":"2024-05-08T08:27:37.231057+0000","last_clean":"2024-05-08T08:27:37.231057+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:37.231057+0000","last_undegraded":"2024-05-08T08:27:37.231057+0000","last_fullsized":"2024-05-08T08:27:37.231057+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":850,"log_dups_size":0,"ondisk_log_size":850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:35:45.798327+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":296,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231165+0000","last_change":"2024-05-08T08:24:56.566341+0000","last_active":"2024-05-08T08:27:37.231165+0000","last_peered":"2024-05-08T08:27:37.231165+0000","last_clean":"2024-05-08T08:27:37.231165+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:37.231165+0000","last_undegraded":"2024-05-08T08:27:37.231165+0000","last_fullsized":"2024-05-08T08:27:37.231165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:46:26.578857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3331,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:45.084361+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:45.084361+0000","last_peered":"2024-05-08T08:27:45.084361+0000","last_clean":"2024-05-08T08:27:45.084361+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:45.084361+0000","last_undegraded":"2024-05-08T08:27:45.084361+0000","last_fullsized":"2024-05-08T08:27:45.084361+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"144'1264","reported_seq":3215,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054597+0000","last_change":"2024-05-08T08:27:23.316843+0000","last_active":"2024-05-08T08:27:38.054597+0000","last_peered":"2024-05-08T08:27:38.054597+0000","last_clean":"2024-05-08T08:27:38.054597+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:38.054597+0000","last_undegraded":"2024-05-08T08:27:38.054597+0000","last_fullsized":"2024-05-08T08:27:38.054597+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":1264,"log_dups_size":0,"ondisk_log_size":1264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":452,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.137864+0000","last_change":"2024-05-08T08:25:45.140614+0000","last_active":"2024-05-08T08:27:38.137864+0000","last_peered":"2024-05-08T08:27:38.137864+0000","last_clean":"2024-05-08T08:27:38.137864+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:38.137864+0000","last_undegraded":"2024-05-08T08:27:38.137864+0000","last_fullsized":"2024-05-08T08:27:38.137864+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:56.551148+0000","last_clean_scrub_stamp":"2024-05-08T08:24:56.551148+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:19:01.624333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":234,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.512171+0000","last_change":"2024-05-08T08:27:05.059497+0000","last_active":"2024-05-08T08:27:38.512171+0000","last_peered":"2024-05-08T08:27:38.512171+0000","last_clean":"2024-05-08T08:27:38.512171+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:38.512171+0000","last_undegraded":"2024-05-08T08:27:38.512171+0000","last_fullsized":"2024-05-08T08:27:38.512171+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:50:56.853131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"136'363","reported_seq":1902,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.512384+0000","last_change":"2024-05-08T08:27:13.120989+0000","last_active":"2024-05-08T08:27:38.512384+0000","last_peered":"2024-05-08T08:27:38.512384+0000","last_clean":"2024-05-08T08:27:38.512384+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:38.512384+0000","last_undegraded":"2024-05-08T08:27:38.512384+0000","last_fullsized":"2024-05-08T08:27:38.512384+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":363,"log_dups_size":0,"ondisk_log_size":363,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:11:32.583458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":297,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.137906+0000","last_change":"2024-05-08T08:24:56.680552+0000","last_active":"2024-05-08T08:27:38.137906+0000","last_peered":"2024-05-08T08:27:38.137906+0000","last_clean":"2024-05-08T08:27:38.137906+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:38.137906+0000","last_undegraded":"2024-05-08T08:27:38.137906+0000","last_fullsized":"2024-05-08T08:27:38.137906+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:07:22.532825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"137'797","reported_seq":6169,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.231454+0000","last_change":"2024-05-08T08:27:13.158703+0000","last_active":"2024-05-08T08:27:37.231454+0000","last_peered":"2024-05-08T08:27:37.231454+0000","last_clean":"2024-05-08T08:27:37.231454+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:37.231454+0000","last_undegraded":"2024-05-08T08:27:37.231454+0000","last_fullsized":"2024-05-08T08:27:37.231454+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":797,"log_dups_size":0,"ondisk_log_size":797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":142,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415518+0000","last_change":"2024-05-08T08:27:29.992916+0000","last_active":"2024-05-08T08:27:37.415518+0000","last_peered":"2024-05-08T08:27:37.415518+0000","last_clean":"2024-05-08T08:27:37.415518+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:37.415518+0000","last_undegraded":"2024-05-08T08:27:37.415518+0000","last_fullsized":"2024-05-08T08:27:37.415518+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:07:54.323209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1976,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232087+0000","last_change":"2024-05-08T08:27:26.167343+0000","last_active":"2024-05-08T08:27:37.232087+0000","last_peered":"2024-05-08T08:27:37.232087+0000","last_clean":"2024-05-08T08:27:37.232087+0000","last_became_active":"2024-05-08T08:27:26.165179+0000","last_became_peered":"2024-05-08T08:27:26.165179+0000","last_unstale":"2024-05-08T08:27:37.232087+0000","last_undegraded":"2024-05-08T08:27:37.232087+0000","last_fullsized":"2024-05-08T08:27:37.232087+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:26:23.290092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":297,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.137870+0000","last_change":"2024-05-08T08:24:56.680605+0000","last_active":"2024-05-08T08:27:38.137870+0000","last_peered":"2024-05-08T08:27:38.137870+0000","last_clean":"2024-05-08T08:27:38.137870+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:38.137870+0000","last_undegraded":"2024-05-08T08:27:38.137870+0000","last_fullsized":"2024-05-08T08:27:38.137870+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:24:55.548970+0000","last_clean_scrub_stamp":"2024-05-08T08:24:55.548970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:17:26.132701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":4994,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232007+0000","last_change":"2024-05-08T08:27:26.163935+0000","last_active":"2024-05-08T08:27:37.232007+0000","last_peered":"2024-05-08T08:27:37.232007+0000","last_clean":"2024-05-08T08:27:37.232007+0000","last_became_active":"2024-05-08T08:27:26.161990+0000","last_became_peered":"2024-05-08T08:27:26.161990+0000","last_unstale":"2024-05-08T08:27:37.232007+0000","last_undegraded":"2024-05-08T08:27:37.232007+0000","last_fullsized":"2024-05-08T08:27:37.232007+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:50:25.301862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"144'353","reported_seq":1591,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.232115+0000","last_change":"2024-05-08T08:27:23.249997+0000","last_active":"2024-05-08T08:27:37.232115+0000","last_peered":"2024-05-08T08:27:37.232115+0000","last_clean":"2024-05-08T08:27:37.232115+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:37.232115+0000","last_undegraded":"2024-05-08T08:27:37.232115+0000","last_fullsized":"2024-05-08T08:27:37.232115+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":353,"log_dups_size":0,"ondisk_log_size":353,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:45:17.630472+0000","scrub_duration":0,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"144'668","reported_seq":1986,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230934+0000","last_change":"2024-05-08T08:27:28.183953+0000","last_active":"2024-05-08T08:27:37.230934+0000","last_peered":"2024-05-08T08:27:37.230934+0000","last_clean":"2024-05-08T08:27:37.230934+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:37.230934+0000","last_undegraded":"2024-05-08T08:27:37.230934+0000","last_fullsized":"2024-05-08T08:27:37.230934+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":668,"log_dups_size":0,"ondisk_log_size":668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:13:51.258724+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"136'368","reported_seq":2574,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.230915+0000","last_change":"2024-05-08T08:27:28.179656+0000","last_active":"2024-05-08T08:27:37.230915+0000","last_peered":"2024-05-08T08:27:37.230915+0000","last_clean":"2024-05-08T08:27:37.230915+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:37.230915+0000","last_undegraded":"2024-05-08T08:27:37.230915+0000","last_fullsized":"2024-05-08T08:27:37.230915+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:06:30.701923+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"130'373","reported_seq":4568,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:37.415547+0000","last_change":"2024-05-08T08:27:29.996890+0000","last_active":"2024-05-08T08:27:37.415547+0000","last_peered":"2024-05-08T08:27:37.415547+0000","last_clean":"2024-05-08T08:27:37.415547+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:37.415547+0000","last_undegraded":"2024-05-08T08:27:37.415547+0000","last_fullsized":"2024-05-08T08:27:37.415547+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:48:32.108598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"136'373","reported_seq":1883,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.138047+0000","last_change":"2024-05-08T08:27:13.115549+0000","last_active":"2024-05-08T08:27:38.138047+0000","last_peered":"2024-05-08T08:27:38.138047+0000","last_clean":"2024-05-08T08:27:38.138047+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:38.138047+0000","last_undegraded":"2024-05-08T08:27:38.138047+0000","last_fullsized":"2024-05-08T08:27:38.138047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":373,"log_dups_size":0,"ondisk_log_size":373,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:23.190217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"134'660","reported_seq":5047,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.138104+0000","last_change":"2024-05-08T08:27:24.170963+0000","last_active":"2024-05-08T08:27:38.138104+0000","last_peered":"2024-05-08T08:27:38.138104+0000","last_clean":"2024-05-08T08:27:38.138104+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:38.138104+0000","last_undegraded":"2024-05-08T08:27:38.138104+0000","last_fullsized":"2024-05-08T08:27:38.138104+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:04:30.030308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"144'457","reported_seq":1927,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.138053+0000","last_change":"2024-05-08T08:27:23.187752+0000","last_active":"2024-05-08T08:27:38.138053+0000","last_peered":"2024-05-08T08:27:38.138053+0000","last_clean":"2024-05-08T08:27:38.138053+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:38.138053+0000","last_undegraded":"2024-05-08T08:27:38.138053+0000","last_fullsized":"2024-05-08T08:27:38.138053+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:50:05.388094+0000","scrub_duration":0,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"137'698","reported_seq":3069,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.511646+0000","last_change":"2024-05-08T08:27:13.117579+0000","last_active":"2024-05-08T08:27:38.511646+0000","last_peered":"2024-05-08T08:27:38.511646+0000","last_clean":"2024-05-08T08:27:38.511646+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:38.511646+0000","last_undegraded":"2024-05-08T08:27:38.511646+0000","last_fullsized":"2024-05-08T08:27:38.511646+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":698,"log_dups_size":0,"ondisk_log_size":698,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:38:05.872143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"144'464","reported_seq":1876,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.137852+0000","last_change":"2024-05-08T08:27:23.343798+0000","last_active":"2024-05-08T08:27:38.137852+0000","last_peered":"2024-05-08T08:27:38.137852+0000","last_clean":"2024-05-08T08:27:38.137852+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:38.137852+0000","last_undegraded":"2024-05-08T08:27:38.137852+0000","last_fullsized":"2024-05-08T08:27:38.137852+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":464,"log_dups_size":0,"ondisk_log_size":464,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:26:34.081134+0000","scrub_duration":0,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"133'375","reported_seq":3463,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.054022+0000","last_change":"2024-05-08T08:27:13.129448+0000","last_active":"2024-05-08T08:27:38.054022+0000","last_peered":"2024-05-08T08:27:38.054022+0000","last_clean":"2024-05-08T08:27:38.054022+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:38.054022+0000","last_undegraded":"2024-05-08T08:27:38.054022+0000","last_fullsized":"2024-05-08T08:27:38.054022+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":375,"log_dups_size":0,"ondisk_log_size":375,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"144'706","reported_seq":4461,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-05-08T08:27:38.138042+0000","last_change":"2024-05-08T08:27:23.275749+0000","last_active":"2024-05-08T08:27:38.138042+0000","last_peered":"2024-05-08T08:27:38.138042+0000","last_clean":"2024-05-08T08:27:38.138042+0000","last_became_active":"2024-05-08T08:27:23.151481+0000","last_became_peered":"2024-05-08T08:27:23.151481+0000","last_unstale":"2024-05-08T08:27:38.138042+0000","last_undegraded":"2024-05-08T08:27:38.138042+0000","last_fullsized":"2024-05-08T08:27:38.138042+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":706,"log_dups_size":0,"ondisk_log_size":706,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:13:45.998918+0000","scrub_duration":0,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2666,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18353,"ondisk_log_size":18353,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574886,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51144,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320696,"statfs":{"total":96636764160,"available":96584392704,"internally_reserved":0,"allocated":1957888,"data_stored":1514735,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":50395594},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574886,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28880,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342960,"statfs":{"total":96636764160,"available":96607191040,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574886,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55248,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316592,"statfs":{"total":96636764160,"available":96580190208,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323335,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35412,"kb_used_data":1860,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336428,"statfs":{"total":96636764160,"available":96600502272,"internally_reserved":0,"allocated":1904640,"data_stored":924386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574887,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51080,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320760,"statfs":{"total":96636764160,"available":96584458240,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":50395591},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574886,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37200,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334640,"statfs":{"total":96636764160,"available":96598671360,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234010,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43084,"kb_used_data":1916,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328756,"statfs":{"total":96636764160,"available":96592646144,"internally_reserved":0,"allocated":1961984,"data_stored":1514754,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574886,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77456,"kb_used_data":1856,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294384,"statfs":{"total":96636764160,"available":96557449216,"internally_reserved":0,"allocated":1900544,"data_stored":924367,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.936 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.937 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T08:24:56.551148+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=56, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T08:24:55.548970+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=24, tm_sec=55, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.938 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.939 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.940 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.940 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:27:47.940 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:28:07.941 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:28:08.254 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:28:08.255 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:28:08.266 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-05-08T08:28:06.456330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18705,"ondisk_log_size":18705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388308,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586412,"statfs":{"total":773094113280,"available":772696485888,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":38,"num_object_clones":0,"num_object_copies":76,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38,"num_whiteouts":38,"num_read":13187,"num_read_kb":91848,"num_write":2465,"num_write_kb":61011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1291,"num_bytes_recovered":5725550,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002798"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5527,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546844+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546844+0000","last_peered":"2024-05-08T08:27:53.546844+0000","last_clean":"2024-05-08T08:27:53.546844+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546844+0000","last_undegraded":"2024-05-08T08:27:53.546844+0000","last_fullsized":"2024-05-08T08:27:53.546844+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3792,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544431+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544431+0000","last_peered":"2024-05-08T08:27:53.544431+0000","last_clean":"2024-05-08T08:27:53.544431+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544431+0000","last_undegraded":"2024-05-08T08:27:53.544431+0000","last_fullsized":"2024-05-08T08:27:53.544431+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2159,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547711+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547711+0000","last_peered":"2024-05-08T08:27:53.547711+0000","last_clean":"2024-05-08T08:27:53.547711+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547711+0000","last_undegraded":"2024-05-08T08:27:53.547711+0000","last_fullsized":"2024-05-08T08:27:53.547711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7701,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547342+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547342+0000","last_peered":"2024-05-08T08:27:53.547342+0000","last_clean":"2024-05-08T08:27:53.547342+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547342+0000","last_undegraded":"2024-05-08T08:27:53.547342+0000","last_fullsized":"2024-05-08T08:27:53.547342+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3063,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550759+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.550759+0000","last_peered":"2024-05-08T08:27:53.550759+0000","last_clean":"2024-05-08T08:27:53.550759+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.550759+0000","last_undegraded":"2024-05-08T08:27:53.550759+0000","last_fullsized":"2024-05-08T08:27:53.550759+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12557,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634730+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634730+0000","last_peered":"2024-05-08T08:27:53.634730+0000","last_clean":"2024-05-08T08:27:53.634730+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634730+0000","last_undegraded":"2024-05-08T08:27:53.634730+0000","last_fullsized":"2024-05-08T08:27:53.634730+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2374,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547163+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547163+0000","last_peered":"2024-05-08T08:27:53.547163+0000","last_clean":"2024-05-08T08:27:53.547163+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547163+0000","last_undegraded":"2024-05-08T08:27:53.547163+0000","last_fullsized":"2024-05-08T08:27:53.547163+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12805,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544766+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544766+0000","last_peered":"2024-05-08T08:27:53.544766+0000","last_clean":"2024-05-08T08:27:53.544766+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544766+0000","last_undegraded":"2024-05-08T08:27:53.544766+0000","last_fullsized":"2024-05-08T08:27:53.544766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551322+0000","last_change":"2024-05-08T08:27:53.551322+0000","last_active":"2024-05-08T08:27:53.551322+0000","last_peered":"2024-05-08T08:27:53.551322+0000","last_clean":"2024-05-08T08:27:53.551322+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:27:53.551322+0000","last_undegraded":"2024-05-08T08:27:53.551322+0000","last_fullsized":"2024-05-08T08:27:53.551322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:06.255241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2153,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547134+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547134+0000","last_peered":"2024-05-08T08:27:53.547134+0000","last_clean":"2024-05-08T08:27:53.547134+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547134+0000","last_undegraded":"2024-05-08T08:27:53.547134+0000","last_fullsized":"2024-05-08T08:27:53.547134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4195,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547443+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547443+0000","last_peered":"2024-05-08T08:27:53.547443+0000","last_clean":"2024-05-08T08:27:53.547443+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547443+0000","last_undegraded":"2024-05-08T08:27:53.547443+0000","last_fullsized":"2024-05-08T08:27:53.547443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550370+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.550370+0000","last_peered":"2024-05-08T08:27:53.550370+0000","last_clean":"2024-05-08T08:27:53.550370+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.550370+0000","last_undegraded":"2024-05-08T08:27:53.550370+0000","last_fullsized":"2024-05-08T08:27:53.550370+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546366+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546366+0000","last_peered":"2024-05-08T08:27:53.546366+0000","last_clean":"2024-05-08T08:27:53.546366+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546366+0000","last_undegraded":"2024-05-08T08:27:53.546366+0000","last_fullsized":"2024-05-08T08:27:53.546366+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9223,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547746+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547746+0000","last_peered":"2024-05-08T08:27:53.547746+0000","last_clean":"2024-05-08T08:27:53.547746+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547746+0000","last_undegraded":"2024-05-08T08:27:53.547746+0000","last_fullsized":"2024-05-08T08:27:53.547746+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546376+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546376+0000","last_peered":"2024-05-08T08:27:53.546376+0000","last_clean":"2024-05-08T08:27:53.546376+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546376+0000","last_undegraded":"2024-05-08T08:27:53.546376+0000","last_fullsized":"2024-05-08T08:27:53.546376+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5738,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547290+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547290+0000","last_peered":"2024-05-08T08:27:53.547290+0000","last_clean":"2024-05-08T08:27:53.547290+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547290+0000","last_undegraded":"2024-05-08T08:27:53.547290+0000","last_fullsized":"2024-05-08T08:27:53.547290+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634636+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634636+0000","last_peered":"2024-05-08T08:27:53.634636+0000","last_clean":"2024-05-08T08:27:53.634636+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634636+0000","last_undegraded":"2024-05-08T08:27:53.634636+0000","last_fullsized":"2024-05-08T08:27:53.634636+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7983,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544782+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544782+0000","last_peered":"2024-05-08T08:27:53.544782+0000","last_clean":"2024-05-08T08:27:53.544782+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544782+0000","last_undegraded":"2024-05-08T08:27:53.544782+0000","last_fullsized":"2024-05-08T08:27:53.544782+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":315,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544923+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544923+0000","last_peered":"2024-05-08T08:27:53.544923+0000","last_clean":"2024-05-08T08:27:53.544923+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544923+0000","last_undegraded":"2024-05-08T08:27:53.544923+0000","last_fullsized":"2024-05-08T08:27:53.544923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3341,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546812+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546812+0000","last_peered":"2024-05-08T08:27:53.546812+0000","last_clean":"2024-05-08T08:27:53.546812+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546812+0000","last_undegraded":"2024-05-08T08:27:53.546812+0000","last_fullsized":"2024-05-08T08:27:53.546812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3246,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546639+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546639+0000","last_peered":"2024-05-08T08:27:53.546639+0000","last_clean":"2024-05-08T08:27:53.546639+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546639+0000","last_undegraded":"2024-05-08T08:27:53.546639+0000","last_fullsized":"2024-05-08T08:27:53.546639+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":473,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546962+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.546962+0000","last_peered":"2024-05-08T08:27:53.546962+0000","last_clean":"2024-05-08T08:27:53.546962+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.546962+0000","last_undegraded":"2024-05-08T08:27:53.546962+0000","last_fullsized":"2024-05-08T08:27:53.546962+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547214+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547214+0000","last_peered":"2024-05-08T08:27:53.547214+0000","last_clean":"2024-05-08T08:27:53.547214+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547214+0000","last_undegraded":"2024-05-08T08:27:53.547214+0000","last_fullsized":"2024-05-08T08:27:53.547214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1932,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547281+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547281+0000","last_peered":"2024-05-08T08:27:53.547281+0000","last_clean":"2024-05-08T08:27:53.547281+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547281+0000","last_undegraded":"2024-05-08T08:27:53.547281+0000","last_fullsized":"2024-05-08T08:27:53.547281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547045+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547045+0000","last_peered":"2024-05-08T08:27:53.547045+0000","last_clean":"2024-05-08T08:27:53.547045+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547045+0000","last_undegraded":"2024-05-08T08:27:53.547045+0000","last_fullsized":"2024-05-08T08:27:53.547045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6195,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544606+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544606+0000","last_peered":"2024-05-08T08:27:53.544606+0000","last_clean":"2024-05-08T08:27:53.544606+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544606+0000","last_undegraded":"2024-05-08T08:27:53.544606+0000","last_fullsized":"2024-05-08T08:27:53.544606+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546949+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.546949+0000","last_peered":"2024-05-08T08:27:53.546949+0000","last_clean":"2024-05-08T08:27:53.546949+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.546949+0000","last_undegraded":"2024-05-08T08:27:53.546949+0000","last_fullsized":"2024-05-08T08:27:53.546949+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1989,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551373+0000","last_change":"2024-05-08T08:27:53.551373+0000","last_active":"2024-05-08T08:27:53.551373+0000","last_peered":"2024-05-08T08:27:53.551373+0000","last_clean":"2024-05-08T08:27:53.551373+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:27:53.551373+0000","last_undegraded":"2024-05-08T08:27:53.551373+0000","last_fullsized":"2024-05-08T08:27:53.551373+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:02.496828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546732+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546732+0000","last_peered":"2024-05-08T08:27:53.546732+0000","last_clean":"2024-05-08T08:27:53.546732+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546732+0000","last_undegraded":"2024-05-08T08:27:53.546732+0000","last_fullsized":"2024-05-08T08:27:53.546732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":5228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.553223+0000","last_change":"2024-05-08T08:27:53.553223+0000","last_active":"2024-05-08T08:27:53.553223+0000","last_peered":"2024-05-08T08:27:53.553223+0000","last_clean":"2024-05-08T08:27:53.553223+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:27:53.553223+0000","last_undegraded":"2024-05-08T08:27:53.553223+0000","last_fullsized":"2024-05-08T08:27:53.553223+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:36.911152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1619,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634643+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634643+0000","last_peered":"2024-05-08T08:27:53.634643+0000","last_clean":"2024-05-08T08:27:53.634643+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634643+0000","last_undegraded":"2024-05-08T08:27:53.634643+0000","last_fullsized":"2024-05-08T08:27:53.634643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2015,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545052+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545052+0000","last_peered":"2024-05-08T08:27:53.545052+0000","last_clean":"2024-05-08T08:27:53.545052+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545052+0000","last_undegraded":"2024-05-08T08:27:53.545052+0000","last_fullsized":"2024-05-08T08:27:53.545052+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2603,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544342+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544342+0000","last_peered":"2024-05-08T08:27:53.544342+0000","last_clean":"2024-05-08T08:27:53.544342+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544342+0000","last_undegraded":"2024-05-08T08:27:53.544342+0000","last_fullsized":"2024-05-08T08:27:53.544342+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4595,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547229+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547229+0000","last_peered":"2024-05-08T08:27:53.547229+0000","last_clean":"2024-05-08T08:27:53.547229+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547229+0000","last_undegraded":"2024-05-08T08:27:53.547229+0000","last_fullsized":"2024-05-08T08:27:53.547229+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1907,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547224+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547224+0000","last_peered":"2024-05-08T08:27:53.547224+0000","last_clean":"2024-05-08T08:27:53.547224+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547224+0000","last_undegraded":"2024-05-08T08:27:53.547224+0000","last_fullsized":"2024-05-08T08:27:53.547224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5077,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285003+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285003+0000","last_peered":"2024-05-08T08:27:54.285003+0000","last_clean":"2024-05-08T08:27:54.285003+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285003+0000","last_undegraded":"2024-05-08T08:27:54.285003+0000","last_fullsized":"2024-05-08T08:27:54.285003+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1952,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547417+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547417+0000","last_peered":"2024-05-08T08:27:53.547417+0000","last_clean":"2024-05-08T08:27:53.547417+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547417+0000","last_undegraded":"2024-05-08T08:27:53.547417+0000","last_fullsized":"2024-05-08T08:27:53.547417+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3090,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546324+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546324+0000","last_peered":"2024-05-08T08:27:53.546324+0000","last_clean":"2024-05-08T08:27:53.546324+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546324+0000","last_undegraded":"2024-05-08T08:27:53.546324+0000","last_fullsized":"2024-05-08T08:27:53.546324+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1903,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547024+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547024+0000","last_peered":"2024-05-08T08:27:53.547024+0000","last_clean":"2024-05-08T08:27:53.547024+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547024+0000","last_undegraded":"2024-05-08T08:27:53.547024+0000","last_fullsized":"2024-05-08T08:27:53.547024+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546685+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546685+0000","last_peered":"2024-05-08T08:27:53.546685+0000","last_clean":"2024-05-08T08:27:53.546685+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546685+0000","last_undegraded":"2024-05-08T08:27:53.546685+0000","last_fullsized":"2024-05-08T08:27:53.546685+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4734,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277362+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277362+0000","last_peered":"2024-05-08T08:27:55.277362+0000","last_clean":"2024-05-08T08:27:55.277362+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277362+0000","last_undegraded":"2024-05-08T08:27:55.277362+0000","last_fullsized":"2024-05-08T08:27:55.277362+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18586,"ondisk_log_size":18586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574890,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574890,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323339,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574891,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574890,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234014,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43148,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328692,"statfs":{"total":96636764160,"available":96592580608,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574890,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77520,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294320,"statfs":{"total":96636764160,"available":96557383680,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:28:08.268 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:08.269 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:08.269 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:08.269 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:28:28.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:28:28.584 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:28:28.584 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:28:28.595 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-05-08T08:28:28.462216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18705,"ondisk_log_size":18705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388308,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586412,"statfs":{"total":773094113280,"available":772696485888,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003412"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5527,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546844+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546844+0000","last_peered":"2024-05-08T08:27:53.546844+0000","last_clean":"2024-05-08T08:27:53.546844+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546844+0000","last_undegraded":"2024-05-08T08:27:53.546844+0000","last_fullsized":"2024-05-08T08:27:53.546844+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3792,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544431+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544431+0000","last_peered":"2024-05-08T08:27:53.544431+0000","last_clean":"2024-05-08T08:27:53.544431+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544431+0000","last_undegraded":"2024-05-08T08:27:53.544431+0000","last_fullsized":"2024-05-08T08:27:53.544431+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2159,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547711+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547711+0000","last_peered":"2024-05-08T08:27:53.547711+0000","last_clean":"2024-05-08T08:27:53.547711+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547711+0000","last_undegraded":"2024-05-08T08:27:53.547711+0000","last_fullsized":"2024-05-08T08:27:53.547711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7701,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547342+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547342+0000","last_peered":"2024-05-08T08:27:53.547342+0000","last_clean":"2024-05-08T08:27:53.547342+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547342+0000","last_undegraded":"2024-05-08T08:27:53.547342+0000","last_fullsized":"2024-05-08T08:27:53.547342+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3063,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550759+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.550759+0000","last_peered":"2024-05-08T08:27:53.550759+0000","last_clean":"2024-05-08T08:27:53.550759+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.550759+0000","last_undegraded":"2024-05-08T08:27:53.550759+0000","last_fullsized":"2024-05-08T08:27:53.550759+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12557,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634730+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634730+0000","last_peered":"2024-05-08T08:27:53.634730+0000","last_clean":"2024-05-08T08:27:53.634730+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634730+0000","last_undegraded":"2024-05-08T08:27:53.634730+0000","last_fullsized":"2024-05-08T08:27:53.634730+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547223+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547223+0000","last_peered":"2024-05-08T08:27:53.547223+0000","last_clean":"2024-05-08T08:27:53.547223+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547223+0000","last_undegraded":"2024-05-08T08:27:53.547223+0000","last_fullsized":"2024-05-08T08:27:53.547223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12805,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544766+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544766+0000","last_peered":"2024-05-08T08:27:53.544766+0000","last_clean":"2024-05-08T08:27:53.544766+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544766+0000","last_undegraded":"2024-05-08T08:27:53.544766+0000","last_fullsized":"2024-05-08T08:27:53.544766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551322+0000","last_change":"2024-05-08T08:27:53.551322+0000","last_active":"2024-05-08T08:27:53.551322+0000","last_peered":"2024-05-08T08:27:53.551322+0000","last_clean":"2024-05-08T08:27:53.551322+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:27:53.551322+0000","last_undegraded":"2024-05-08T08:27:53.551322+0000","last_fullsized":"2024-05-08T08:27:53.551322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:06.255241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2153,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547134+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547134+0000","last_peered":"2024-05-08T08:27:53.547134+0000","last_clean":"2024-05-08T08:27:53.547134+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547134+0000","last_undegraded":"2024-05-08T08:27:53.547134+0000","last_fullsized":"2024-05-08T08:27:53.547134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4195,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547443+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547443+0000","last_peered":"2024-05-08T08:27:53.547443+0000","last_clean":"2024-05-08T08:27:53.547443+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547443+0000","last_undegraded":"2024-05-08T08:27:53.547443+0000","last_fullsized":"2024-05-08T08:27:53.547443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550370+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.550370+0000","last_peered":"2024-05-08T08:27:53.550370+0000","last_clean":"2024-05-08T08:27:53.550370+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.550370+0000","last_undegraded":"2024-05-08T08:27:53.550370+0000","last_fullsized":"2024-05-08T08:27:53.550370+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546366+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546366+0000","last_peered":"2024-05-08T08:27:53.546366+0000","last_clean":"2024-05-08T08:27:53.546366+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546366+0000","last_undegraded":"2024-05-08T08:27:53.546366+0000","last_fullsized":"2024-05-08T08:27:53.546366+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9223,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547746+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547746+0000","last_peered":"2024-05-08T08:27:53.547746+0000","last_clean":"2024-05-08T08:27:53.547746+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547746+0000","last_undegraded":"2024-05-08T08:27:53.547746+0000","last_fullsized":"2024-05-08T08:27:53.547746+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546376+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546376+0000","last_peered":"2024-05-08T08:27:53.546376+0000","last_clean":"2024-05-08T08:27:53.546376+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546376+0000","last_undegraded":"2024-05-08T08:27:53.546376+0000","last_fullsized":"2024-05-08T08:27:53.546376+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5738,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547290+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547290+0000","last_peered":"2024-05-08T08:27:53.547290+0000","last_clean":"2024-05-08T08:27:53.547290+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547290+0000","last_undegraded":"2024-05-08T08:27:53.547290+0000","last_fullsized":"2024-05-08T08:27:53.547290+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634636+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634636+0000","last_peered":"2024-05-08T08:27:53.634636+0000","last_clean":"2024-05-08T08:27:53.634636+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634636+0000","last_undegraded":"2024-05-08T08:27:53.634636+0000","last_fullsized":"2024-05-08T08:27:53.634636+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7983,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544782+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544782+0000","last_peered":"2024-05-08T08:27:53.544782+0000","last_clean":"2024-05-08T08:27:53.544782+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544782+0000","last_undegraded":"2024-05-08T08:27:53.544782+0000","last_fullsized":"2024-05-08T08:27:53.544782+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":315,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544923+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544923+0000","last_peered":"2024-05-08T08:27:53.544923+0000","last_clean":"2024-05-08T08:27:53.544923+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544923+0000","last_undegraded":"2024-05-08T08:27:53.544923+0000","last_fullsized":"2024-05-08T08:27:53.544923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3341,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546812+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546812+0000","last_peered":"2024-05-08T08:27:53.546812+0000","last_clean":"2024-05-08T08:27:53.546812+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546812+0000","last_undegraded":"2024-05-08T08:27:53.546812+0000","last_fullsized":"2024-05-08T08:27:53.546812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3246,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546639+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546639+0000","last_peered":"2024-05-08T08:27:53.546639+0000","last_clean":"2024-05-08T08:27:53.546639+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546639+0000","last_undegraded":"2024-05-08T08:27:53.546639+0000","last_fullsized":"2024-05-08T08:27:53.546639+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":473,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546962+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.546962+0000","last_peered":"2024-05-08T08:27:53.546962+0000","last_clean":"2024-05-08T08:27:53.546962+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.546962+0000","last_undegraded":"2024-05-08T08:27:53.546962+0000","last_fullsized":"2024-05-08T08:27:53.546962+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547214+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547214+0000","last_peered":"2024-05-08T08:27:53.547214+0000","last_clean":"2024-05-08T08:27:53.547214+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547214+0000","last_undegraded":"2024-05-08T08:27:53.547214+0000","last_fullsized":"2024-05-08T08:27:53.547214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1932,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547281+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547281+0000","last_peered":"2024-05-08T08:27:53.547281+0000","last_clean":"2024-05-08T08:27:53.547281+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547281+0000","last_undegraded":"2024-05-08T08:27:53.547281+0000","last_fullsized":"2024-05-08T08:27:53.547281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547045+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547045+0000","last_peered":"2024-05-08T08:27:53.547045+0000","last_clean":"2024-05-08T08:27:53.547045+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547045+0000","last_undegraded":"2024-05-08T08:27:53.547045+0000","last_fullsized":"2024-05-08T08:27:53.547045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6195,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544606+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544606+0000","last_peered":"2024-05-08T08:27:53.544606+0000","last_clean":"2024-05-08T08:27:53.544606+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544606+0000","last_undegraded":"2024-05-08T08:27:53.544606+0000","last_fullsized":"2024-05-08T08:27:53.544606+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546949+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.546949+0000","last_peered":"2024-05-08T08:27:53.546949+0000","last_clean":"2024-05-08T08:27:53.546949+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.546949+0000","last_undegraded":"2024-05-08T08:27:53.546949+0000","last_fullsized":"2024-05-08T08:27:53.546949+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1989,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551373+0000","last_change":"2024-05-08T08:27:53.551373+0000","last_active":"2024-05-08T08:27:53.551373+0000","last_peered":"2024-05-08T08:27:53.551373+0000","last_clean":"2024-05-08T08:27:53.551373+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:27:53.551373+0000","last_undegraded":"2024-05-08T08:27:53.551373+0000","last_fullsized":"2024-05-08T08:27:53.551373+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:02.496828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546732+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546732+0000","last_peered":"2024-05-08T08:27:53.546732+0000","last_clean":"2024-05-08T08:27:53.546732+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546732+0000","last_undegraded":"2024-05-08T08:27:53.546732+0000","last_fullsized":"2024-05-08T08:27:53.546732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":5228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.553223+0000","last_change":"2024-05-08T08:27:53.553223+0000","last_active":"2024-05-08T08:27:53.553223+0000","last_peered":"2024-05-08T08:27:53.553223+0000","last_clean":"2024-05-08T08:27:53.553223+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:27:53.553223+0000","last_undegraded":"2024-05-08T08:27:53.553223+0000","last_fullsized":"2024-05-08T08:27:53.553223+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:36.911152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1619,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634643+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634643+0000","last_peered":"2024-05-08T08:27:53.634643+0000","last_clean":"2024-05-08T08:27:53.634643+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634643+0000","last_undegraded":"2024-05-08T08:27:53.634643+0000","last_fullsized":"2024-05-08T08:27:53.634643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2016,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545099+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545099+0000","last_peered":"2024-05-08T08:27:53.545099+0000","last_clean":"2024-05-08T08:27:53.545099+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545099+0000","last_undegraded":"2024-05-08T08:27:53.545099+0000","last_fullsized":"2024-05-08T08:27:53.545099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2603,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544342+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544342+0000","last_peered":"2024-05-08T08:27:53.544342+0000","last_clean":"2024-05-08T08:27:53.544342+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544342+0000","last_undegraded":"2024-05-08T08:27:53.544342+0000","last_fullsized":"2024-05-08T08:27:53.544342+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4595,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547229+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547229+0000","last_peered":"2024-05-08T08:27:53.547229+0000","last_clean":"2024-05-08T08:27:53.547229+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547229+0000","last_undegraded":"2024-05-08T08:27:53.547229+0000","last_fullsized":"2024-05-08T08:27:53.547229+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1907,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547224+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547224+0000","last_peered":"2024-05-08T08:27:53.547224+0000","last_clean":"2024-05-08T08:27:53.547224+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547224+0000","last_undegraded":"2024-05-08T08:27:53.547224+0000","last_fullsized":"2024-05-08T08:27:53.547224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5077,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285003+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285003+0000","last_peered":"2024-05-08T08:27:54.285003+0000","last_clean":"2024-05-08T08:27:54.285003+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285003+0000","last_undegraded":"2024-05-08T08:27:54.285003+0000","last_fullsized":"2024-05-08T08:27:54.285003+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1952,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547417+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547417+0000","last_peered":"2024-05-08T08:27:53.547417+0000","last_clean":"2024-05-08T08:27:53.547417+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547417+0000","last_undegraded":"2024-05-08T08:27:53.547417+0000","last_fullsized":"2024-05-08T08:27:53.547417+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3090,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546324+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546324+0000","last_peered":"2024-05-08T08:27:53.546324+0000","last_clean":"2024-05-08T08:27:53.546324+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546324+0000","last_undegraded":"2024-05-08T08:27:53.546324+0000","last_fullsized":"2024-05-08T08:27:53.546324+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1903,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547024+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547024+0000","last_peered":"2024-05-08T08:27:53.547024+0000","last_clean":"2024-05-08T08:27:53.547024+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547024+0000","last_undegraded":"2024-05-08T08:27:53.547024+0000","last_fullsized":"2024-05-08T08:27:53.547024+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546685+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546685+0000","last_peered":"2024-05-08T08:27:53.546685+0000","last_clean":"2024-05-08T08:27:53.546685+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546685+0000","last_undegraded":"2024-05-08T08:27:53.546685+0000","last_fullsized":"2024-05-08T08:27:53.546685+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4734,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277362+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277362+0000","last_peered":"2024-05-08T08:27:55.277362+0000","last_clean":"2024-05-08T08:27:55.277362+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277362+0000","last_undegraded":"2024-05-08T08:27:55.277362+0000","last_fullsized":"2024-05-08T08:27:55.277362+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18586,"ondisk_log_size":18586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323343,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234019,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43148,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328692,"statfs":{"total":96636764160,"available":96592580608,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77520,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294320,"statfs":{"total":96636764160,"available":96557383680,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:28:28.598 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:28.598 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:28.598 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:28.598 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:28:48.600 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:28:48.915 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:28:48.915 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:28:48.927 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":273,"stamp":"2024-05-08T08:28:48.466552+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18705,"ondisk_log_size":18705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388308,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586412,"statfs":{"total":773094113280,"available":772696485888,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002631"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5527,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546844+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546844+0000","last_peered":"2024-05-08T08:27:53.546844+0000","last_clean":"2024-05-08T08:27:53.546844+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546844+0000","last_undegraded":"2024-05-08T08:27:53.546844+0000","last_fullsized":"2024-05-08T08:27:53.546844+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3792,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544431+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544431+0000","last_peered":"2024-05-08T08:27:53.544431+0000","last_clean":"2024-05-08T08:27:53.544431+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544431+0000","last_undegraded":"2024-05-08T08:27:53.544431+0000","last_fullsized":"2024-05-08T08:27:53.544431+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2159,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547711+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547711+0000","last_peered":"2024-05-08T08:27:53.547711+0000","last_clean":"2024-05-08T08:27:53.547711+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547711+0000","last_undegraded":"2024-05-08T08:27:53.547711+0000","last_fullsized":"2024-05-08T08:27:53.547711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7701,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547342+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547342+0000","last_peered":"2024-05-08T08:27:53.547342+0000","last_clean":"2024-05-08T08:27:53.547342+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547342+0000","last_undegraded":"2024-05-08T08:27:53.547342+0000","last_fullsized":"2024-05-08T08:27:53.547342+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3063,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550759+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.550759+0000","last_peered":"2024-05-08T08:27:53.550759+0000","last_clean":"2024-05-08T08:27:53.550759+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.550759+0000","last_undegraded":"2024-05-08T08:27:53.550759+0000","last_fullsized":"2024-05-08T08:27:53.550759+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12557,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634730+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634730+0000","last_peered":"2024-05-08T08:27:53.634730+0000","last_clean":"2024-05-08T08:27:53.634730+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634730+0000","last_undegraded":"2024-05-08T08:27:53.634730+0000","last_fullsized":"2024-05-08T08:27:53.634730+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547223+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547223+0000","last_peered":"2024-05-08T08:27:53.547223+0000","last_clean":"2024-05-08T08:27:53.547223+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547223+0000","last_undegraded":"2024-05-08T08:27:53.547223+0000","last_fullsized":"2024-05-08T08:27:53.547223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12805,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544766+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544766+0000","last_peered":"2024-05-08T08:27:53.544766+0000","last_clean":"2024-05-08T08:27:53.544766+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544766+0000","last_undegraded":"2024-05-08T08:27:53.544766+0000","last_fullsized":"2024-05-08T08:27:53.544766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551322+0000","last_change":"2024-05-08T08:27:53.551322+0000","last_active":"2024-05-08T08:27:53.551322+0000","last_peered":"2024-05-08T08:27:53.551322+0000","last_clean":"2024-05-08T08:27:53.551322+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:27:53.551322+0000","last_undegraded":"2024-05-08T08:27:53.551322+0000","last_fullsized":"2024-05-08T08:27:53.551322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:06.255241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2153,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547134+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547134+0000","last_peered":"2024-05-08T08:27:53.547134+0000","last_clean":"2024-05-08T08:27:53.547134+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547134+0000","last_undegraded":"2024-05-08T08:27:53.547134+0000","last_fullsized":"2024-05-08T08:27:53.547134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4195,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547443+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547443+0000","last_peered":"2024-05-08T08:27:53.547443+0000","last_clean":"2024-05-08T08:27:53.547443+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547443+0000","last_undegraded":"2024-05-08T08:27:53.547443+0000","last_fullsized":"2024-05-08T08:27:53.547443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550370+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.550370+0000","last_peered":"2024-05-08T08:27:53.550370+0000","last_clean":"2024-05-08T08:27:53.550370+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.550370+0000","last_undegraded":"2024-05-08T08:27:53.550370+0000","last_fullsized":"2024-05-08T08:27:53.550370+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546366+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546366+0000","last_peered":"2024-05-08T08:27:53.546366+0000","last_clean":"2024-05-08T08:27:53.546366+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546366+0000","last_undegraded":"2024-05-08T08:27:53.546366+0000","last_fullsized":"2024-05-08T08:27:53.546366+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9223,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547746+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547746+0000","last_peered":"2024-05-08T08:27:53.547746+0000","last_clean":"2024-05-08T08:27:53.547746+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547746+0000","last_undegraded":"2024-05-08T08:27:53.547746+0000","last_fullsized":"2024-05-08T08:27:53.547746+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546376+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546376+0000","last_peered":"2024-05-08T08:27:53.546376+0000","last_clean":"2024-05-08T08:27:53.546376+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546376+0000","last_undegraded":"2024-05-08T08:27:53.546376+0000","last_fullsized":"2024-05-08T08:27:53.546376+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5738,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547290+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547290+0000","last_peered":"2024-05-08T08:27:53.547290+0000","last_clean":"2024-05-08T08:27:53.547290+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547290+0000","last_undegraded":"2024-05-08T08:27:53.547290+0000","last_fullsized":"2024-05-08T08:27:53.547290+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634636+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634636+0000","last_peered":"2024-05-08T08:27:53.634636+0000","last_clean":"2024-05-08T08:27:53.634636+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634636+0000","last_undegraded":"2024-05-08T08:27:53.634636+0000","last_fullsized":"2024-05-08T08:27:53.634636+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7983,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544782+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544782+0000","last_peered":"2024-05-08T08:27:53.544782+0000","last_clean":"2024-05-08T08:27:53.544782+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544782+0000","last_undegraded":"2024-05-08T08:27:53.544782+0000","last_fullsized":"2024-05-08T08:27:53.544782+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":315,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544923+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544923+0000","last_peered":"2024-05-08T08:27:53.544923+0000","last_clean":"2024-05-08T08:27:53.544923+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544923+0000","last_undegraded":"2024-05-08T08:27:53.544923+0000","last_fullsized":"2024-05-08T08:27:53.544923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3341,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546812+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546812+0000","last_peered":"2024-05-08T08:27:53.546812+0000","last_clean":"2024-05-08T08:27:53.546812+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546812+0000","last_undegraded":"2024-05-08T08:27:53.546812+0000","last_fullsized":"2024-05-08T08:27:53.546812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3246,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546639+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546639+0000","last_peered":"2024-05-08T08:27:53.546639+0000","last_clean":"2024-05-08T08:27:53.546639+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546639+0000","last_undegraded":"2024-05-08T08:27:53.546639+0000","last_fullsized":"2024-05-08T08:27:53.546639+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":473,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546962+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.546962+0000","last_peered":"2024-05-08T08:27:53.546962+0000","last_clean":"2024-05-08T08:27:53.546962+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.546962+0000","last_undegraded":"2024-05-08T08:27:53.546962+0000","last_fullsized":"2024-05-08T08:27:53.546962+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547214+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547214+0000","last_peered":"2024-05-08T08:27:53.547214+0000","last_clean":"2024-05-08T08:27:53.547214+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547214+0000","last_undegraded":"2024-05-08T08:27:53.547214+0000","last_fullsized":"2024-05-08T08:27:53.547214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1932,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547281+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547281+0000","last_peered":"2024-05-08T08:27:53.547281+0000","last_clean":"2024-05-08T08:27:53.547281+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547281+0000","last_undegraded":"2024-05-08T08:27:53.547281+0000","last_fullsized":"2024-05-08T08:27:53.547281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547045+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547045+0000","last_peered":"2024-05-08T08:27:53.547045+0000","last_clean":"2024-05-08T08:27:53.547045+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547045+0000","last_undegraded":"2024-05-08T08:27:53.547045+0000","last_fullsized":"2024-05-08T08:27:53.547045+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544667+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544667+0000","last_peered":"2024-05-08T08:27:53.544667+0000","last_clean":"2024-05-08T08:27:53.544667+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544667+0000","last_undegraded":"2024-05-08T08:27:53.544667+0000","last_fullsized":"2024-05-08T08:27:53.544667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546949+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.546949+0000","last_peered":"2024-05-08T08:27:53.546949+0000","last_clean":"2024-05-08T08:27:53.546949+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.546949+0000","last_undegraded":"2024-05-08T08:27:53.546949+0000","last_fullsized":"2024-05-08T08:27:53.546949+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1989,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551373+0000","last_change":"2024-05-08T08:27:53.551373+0000","last_active":"2024-05-08T08:27:53.551373+0000","last_peered":"2024-05-08T08:27:53.551373+0000","last_clean":"2024-05-08T08:27:53.551373+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:27:53.551373+0000","last_undegraded":"2024-05-08T08:27:53.551373+0000","last_fullsized":"2024-05-08T08:27:53.551373+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:02.496828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546732+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546732+0000","last_peered":"2024-05-08T08:27:53.546732+0000","last_clean":"2024-05-08T08:27:53.546732+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546732+0000","last_undegraded":"2024-05-08T08:27:53.546732+0000","last_fullsized":"2024-05-08T08:27:53.546732+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":5228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.553223+0000","last_change":"2024-05-08T08:27:53.553223+0000","last_active":"2024-05-08T08:27:53.553223+0000","last_peered":"2024-05-08T08:27:53.553223+0000","last_clean":"2024-05-08T08:27:53.553223+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:27:53.553223+0000","last_undegraded":"2024-05-08T08:27:53.553223+0000","last_fullsized":"2024-05-08T08:27:53.553223+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:36.911152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1619,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634643+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634643+0000","last_peered":"2024-05-08T08:27:53.634643+0000","last_clean":"2024-05-08T08:27:53.634643+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634643+0000","last_undegraded":"2024-05-08T08:27:53.634643+0000","last_fullsized":"2024-05-08T08:27:53.634643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2016,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545099+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545099+0000","last_peered":"2024-05-08T08:27:53.545099+0000","last_clean":"2024-05-08T08:27:53.545099+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545099+0000","last_undegraded":"2024-05-08T08:27:53.545099+0000","last_fullsized":"2024-05-08T08:27:53.545099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2603,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544342+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544342+0000","last_peered":"2024-05-08T08:27:53.544342+0000","last_clean":"2024-05-08T08:27:53.544342+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544342+0000","last_undegraded":"2024-05-08T08:27:53.544342+0000","last_fullsized":"2024-05-08T08:27:53.544342+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4595,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547229+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547229+0000","last_peered":"2024-05-08T08:27:53.547229+0000","last_clean":"2024-05-08T08:27:53.547229+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547229+0000","last_undegraded":"2024-05-08T08:27:53.547229+0000","last_fullsized":"2024-05-08T08:27:53.547229+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1907,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547224+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547224+0000","last_peered":"2024-05-08T08:27:53.547224+0000","last_clean":"2024-05-08T08:27:53.547224+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547224+0000","last_undegraded":"2024-05-08T08:27:53.547224+0000","last_fullsized":"2024-05-08T08:27:53.547224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5077,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285003+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285003+0000","last_peered":"2024-05-08T08:27:54.285003+0000","last_clean":"2024-05-08T08:27:54.285003+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285003+0000","last_undegraded":"2024-05-08T08:27:54.285003+0000","last_fullsized":"2024-05-08T08:27:54.285003+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1952,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547417+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547417+0000","last_peered":"2024-05-08T08:27:53.547417+0000","last_clean":"2024-05-08T08:27:53.547417+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547417+0000","last_undegraded":"2024-05-08T08:27:53.547417+0000","last_fullsized":"2024-05-08T08:27:53.547417+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3090,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546324+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546324+0000","last_peered":"2024-05-08T08:27:53.546324+0000","last_clean":"2024-05-08T08:27:53.546324+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546324+0000","last_undegraded":"2024-05-08T08:27:53.546324+0000","last_fullsized":"2024-05-08T08:27:53.546324+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1903,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547024+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547024+0000","last_peered":"2024-05-08T08:27:53.547024+0000","last_clean":"2024-05-08T08:27:53.547024+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547024+0000","last_undegraded":"2024-05-08T08:27:53.547024+0000","last_fullsized":"2024-05-08T08:27:53.547024+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546685+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546685+0000","last_peered":"2024-05-08T08:27:53.546685+0000","last_clean":"2024-05-08T08:27:53.546685+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546685+0000","last_undegraded":"2024-05-08T08:27:53.546685+0000","last_fullsized":"2024-05-08T08:27:53.546685+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4734,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277362+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277362+0000","last_peered":"2024-05-08T08:27:55.277362+0000","last_clean":"2024-05-08T08:27:55.277362+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277362+0000","last_undegraded":"2024-05-08T08:27:55.277362+0000","last_fullsized":"2024-05-08T08:27:55.277362+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18586,"ondisk_log_size":18586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574898,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574898,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574898,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323347,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234023,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43148,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328692,"statfs":{"total":96636764160,"available":96592580608,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574898,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77520,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294320,"statfs":{"total":96636764160,"available":96557383680,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:28:48.929 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:48.929 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:48.930 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:28:48.930 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:29:08.931 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:29:09.245 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:29:09.245 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:29:09.257 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":283,"stamp":"2024-05-08T08:29:08.471162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18705,"ondisk_log_size":18705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388308,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586412,"statfs":{"total":773094113280,"available":772696485888,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002979"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5527,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546844+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546844+0000","last_peered":"2024-05-08T08:27:53.546844+0000","last_clean":"2024-05-08T08:27:53.546844+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546844+0000","last_undegraded":"2024-05-08T08:27:53.546844+0000","last_fullsized":"2024-05-08T08:27:53.546844+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3792,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544431+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544431+0000","last_peered":"2024-05-08T08:27:53.544431+0000","last_clean":"2024-05-08T08:27:53.544431+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544431+0000","last_undegraded":"2024-05-08T08:27:53.544431+0000","last_fullsized":"2024-05-08T08:27:53.544431+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2159,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547711+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547711+0000","last_peered":"2024-05-08T08:27:53.547711+0000","last_clean":"2024-05-08T08:27:53.547711+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547711+0000","last_undegraded":"2024-05-08T08:27:53.547711+0000","last_fullsized":"2024-05-08T08:27:53.547711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7701,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547342+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547342+0000","last_peered":"2024-05-08T08:27:53.547342+0000","last_clean":"2024-05-08T08:27:53.547342+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547342+0000","last_undegraded":"2024-05-08T08:27:53.547342+0000","last_fullsized":"2024-05-08T08:27:53.547342+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3063,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550759+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.550759+0000","last_peered":"2024-05-08T08:27:53.550759+0000","last_clean":"2024-05-08T08:27:53.550759+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.550759+0000","last_undegraded":"2024-05-08T08:27:53.550759+0000","last_fullsized":"2024-05-08T08:27:53.550759+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12557,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634730+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634730+0000","last_peered":"2024-05-08T08:27:53.634730+0000","last_clean":"2024-05-08T08:27:53.634730+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634730+0000","last_undegraded":"2024-05-08T08:27:53.634730+0000","last_fullsized":"2024-05-08T08:27:53.634730+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547223+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547223+0000","last_peered":"2024-05-08T08:27:53.547223+0000","last_clean":"2024-05-08T08:27:53.547223+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547223+0000","last_undegraded":"2024-05-08T08:27:53.547223+0000","last_fullsized":"2024-05-08T08:27:53.547223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12805,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544766+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544766+0000","last_peered":"2024-05-08T08:27:53.544766+0000","last_clean":"2024-05-08T08:27:53.544766+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544766+0000","last_undegraded":"2024-05-08T08:27:53.544766+0000","last_fullsized":"2024-05-08T08:27:53.544766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551322+0000","last_change":"2024-05-08T08:27:53.551322+0000","last_active":"2024-05-08T08:27:53.551322+0000","last_peered":"2024-05-08T08:27:53.551322+0000","last_clean":"2024-05-08T08:27:53.551322+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:27:53.551322+0000","last_undegraded":"2024-05-08T08:27:53.551322+0000","last_fullsized":"2024-05-08T08:27:53.551322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:06.255241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2153,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547134+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547134+0000","last_peered":"2024-05-08T08:27:53.547134+0000","last_clean":"2024-05-08T08:27:53.547134+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547134+0000","last_undegraded":"2024-05-08T08:27:53.547134+0000","last_fullsized":"2024-05-08T08:27:53.547134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4195,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547443+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547443+0000","last_peered":"2024-05-08T08:27:53.547443+0000","last_clean":"2024-05-08T08:27:53.547443+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547443+0000","last_undegraded":"2024-05-08T08:27:53.547443+0000","last_fullsized":"2024-05-08T08:27:53.547443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550370+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.550370+0000","last_peered":"2024-05-08T08:27:53.550370+0000","last_clean":"2024-05-08T08:27:53.550370+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.550370+0000","last_undegraded":"2024-05-08T08:27:53.550370+0000","last_fullsized":"2024-05-08T08:27:53.550370+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546366+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546366+0000","last_peered":"2024-05-08T08:27:53.546366+0000","last_clean":"2024-05-08T08:27:53.546366+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546366+0000","last_undegraded":"2024-05-08T08:27:53.546366+0000","last_fullsized":"2024-05-08T08:27:53.546366+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9223,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547746+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547746+0000","last_peered":"2024-05-08T08:27:53.547746+0000","last_clean":"2024-05-08T08:27:53.547746+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547746+0000","last_undegraded":"2024-05-08T08:27:53.547746+0000","last_fullsized":"2024-05-08T08:27:53.547746+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546376+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546376+0000","last_peered":"2024-05-08T08:27:53.546376+0000","last_clean":"2024-05-08T08:27:53.546376+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546376+0000","last_undegraded":"2024-05-08T08:27:53.546376+0000","last_fullsized":"2024-05-08T08:27:53.546376+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5738,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547290+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547290+0000","last_peered":"2024-05-08T08:27:53.547290+0000","last_clean":"2024-05-08T08:27:53.547290+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547290+0000","last_undegraded":"2024-05-08T08:27:53.547290+0000","last_fullsized":"2024-05-08T08:27:53.547290+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634636+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634636+0000","last_peered":"2024-05-08T08:27:53.634636+0000","last_clean":"2024-05-08T08:27:53.634636+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634636+0000","last_undegraded":"2024-05-08T08:27:53.634636+0000","last_fullsized":"2024-05-08T08:27:53.634636+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7983,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544782+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544782+0000","last_peered":"2024-05-08T08:27:53.544782+0000","last_clean":"2024-05-08T08:27:53.544782+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544782+0000","last_undegraded":"2024-05-08T08:27:53.544782+0000","last_fullsized":"2024-05-08T08:27:53.544782+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":315,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544923+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544923+0000","last_peered":"2024-05-08T08:27:53.544923+0000","last_clean":"2024-05-08T08:27:53.544923+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544923+0000","last_undegraded":"2024-05-08T08:27:53.544923+0000","last_fullsized":"2024-05-08T08:27:53.544923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3341,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546812+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546812+0000","last_peered":"2024-05-08T08:27:53.546812+0000","last_clean":"2024-05-08T08:27:53.546812+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546812+0000","last_undegraded":"2024-05-08T08:27:53.546812+0000","last_fullsized":"2024-05-08T08:27:53.546812+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3246,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546639+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546639+0000","last_peered":"2024-05-08T08:27:53.546639+0000","last_clean":"2024-05-08T08:27:53.546639+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546639+0000","last_undegraded":"2024-05-08T08:27:53.546639+0000","last_fullsized":"2024-05-08T08:27:53.546639+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":474,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547156+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.547156+0000","last_peered":"2024-05-08T08:27:53.547156+0000","last_clean":"2024-05-08T08:27:53.547156+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.547156+0000","last_undegraded":"2024-05-08T08:27:53.547156+0000","last_fullsized":"2024-05-08T08:27:53.547156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547214+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547214+0000","last_peered":"2024-05-08T08:27:53.547214+0000","last_clean":"2024-05-08T08:27:53.547214+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547214+0000","last_undegraded":"2024-05-08T08:27:53.547214+0000","last_fullsized":"2024-05-08T08:27:53.547214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1932,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547281+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547281+0000","last_peered":"2024-05-08T08:27:53.547281+0000","last_clean":"2024-05-08T08:27:53.547281+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547281+0000","last_undegraded":"2024-05-08T08:27:53.547281+0000","last_fullsized":"2024-05-08T08:27:53.547281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547087+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547087+0000","last_peered":"2024-05-08T08:27:53.547087+0000","last_clean":"2024-05-08T08:27:53.547087+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547087+0000","last_undegraded":"2024-05-08T08:27:53.547087+0000","last_fullsized":"2024-05-08T08:27:53.547087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544667+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544667+0000","last_peered":"2024-05-08T08:27:53.544667+0000","last_clean":"2024-05-08T08:27:53.544667+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544667+0000","last_undegraded":"2024-05-08T08:27:53.544667+0000","last_fullsized":"2024-05-08T08:27:53.544667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546949+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.546949+0000","last_peered":"2024-05-08T08:27:53.546949+0000","last_clean":"2024-05-08T08:27:53.546949+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.546949+0000","last_undegraded":"2024-05-08T08:27:53.546949+0000","last_fullsized":"2024-05-08T08:27:53.546949+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1989,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551373+0000","last_change":"2024-05-08T08:27:53.551373+0000","last_active":"2024-05-08T08:27:53.551373+0000","last_peered":"2024-05-08T08:27:53.551373+0000","last_clean":"2024-05-08T08:27:53.551373+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:27:53.551373+0000","last_undegraded":"2024-05-08T08:27:53.551373+0000","last_fullsized":"2024-05-08T08:27:53.551373+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:02.496828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546796+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546796+0000","last_peered":"2024-05-08T08:27:53.546796+0000","last_clean":"2024-05-08T08:27:53.546796+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546796+0000","last_undegraded":"2024-05-08T08:27:53.546796+0000","last_fullsized":"2024-05-08T08:27:53.546796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":5228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.553223+0000","last_change":"2024-05-08T08:27:53.553223+0000","last_active":"2024-05-08T08:27:53.553223+0000","last_peered":"2024-05-08T08:27:53.553223+0000","last_clean":"2024-05-08T08:27:53.553223+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:27:53.553223+0000","last_undegraded":"2024-05-08T08:27:53.553223+0000","last_fullsized":"2024-05-08T08:27:53.553223+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:36.911152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1619,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634643+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634643+0000","last_peered":"2024-05-08T08:27:53.634643+0000","last_clean":"2024-05-08T08:27:53.634643+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634643+0000","last_undegraded":"2024-05-08T08:27:53.634643+0000","last_fullsized":"2024-05-08T08:27:53.634643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2016,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545099+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545099+0000","last_peered":"2024-05-08T08:27:53.545099+0000","last_clean":"2024-05-08T08:27:53.545099+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545099+0000","last_undegraded":"2024-05-08T08:27:53.545099+0000","last_fullsized":"2024-05-08T08:27:53.545099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2603,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544342+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544342+0000","last_peered":"2024-05-08T08:27:53.544342+0000","last_clean":"2024-05-08T08:27:53.544342+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544342+0000","last_undegraded":"2024-05-08T08:27:53.544342+0000","last_fullsized":"2024-05-08T08:27:53.544342+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4595,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547229+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547229+0000","last_peered":"2024-05-08T08:27:53.547229+0000","last_clean":"2024-05-08T08:27:53.547229+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547229+0000","last_undegraded":"2024-05-08T08:27:53.547229+0000","last_fullsized":"2024-05-08T08:27:53.547229+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1907,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547224+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547224+0000","last_peered":"2024-05-08T08:27:53.547224+0000","last_clean":"2024-05-08T08:27:53.547224+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547224+0000","last_undegraded":"2024-05-08T08:27:53.547224+0000","last_fullsized":"2024-05-08T08:27:53.547224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5077,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285003+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285003+0000","last_peered":"2024-05-08T08:27:54.285003+0000","last_clean":"2024-05-08T08:27:54.285003+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285003+0000","last_undegraded":"2024-05-08T08:27:54.285003+0000","last_fullsized":"2024-05-08T08:27:54.285003+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1952,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547417+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547417+0000","last_peered":"2024-05-08T08:27:53.547417+0000","last_clean":"2024-05-08T08:27:53.547417+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547417+0000","last_undegraded":"2024-05-08T08:27:53.547417+0000","last_fullsized":"2024-05-08T08:27:53.547417+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3090,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546324+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546324+0000","last_peered":"2024-05-08T08:27:53.546324+0000","last_clean":"2024-05-08T08:27:53.546324+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546324+0000","last_undegraded":"2024-05-08T08:27:53.546324+0000","last_fullsized":"2024-05-08T08:27:53.546324+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1903,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547024+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547024+0000","last_peered":"2024-05-08T08:27:53.547024+0000","last_clean":"2024-05-08T08:27:53.547024+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547024+0000","last_undegraded":"2024-05-08T08:27:53.547024+0000","last_fullsized":"2024-05-08T08:27:53.547024+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546685+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546685+0000","last_peered":"2024-05-08T08:27:53.546685+0000","last_clean":"2024-05-08T08:27:53.546685+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546685+0000","last_undegraded":"2024-05-08T08:27:53.546685+0000","last_fullsized":"2024-05-08T08:27:53.546685+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4734,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277362+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277362+0000","last_peered":"2024-05-08T08:27:55.277362+0000","last_clean":"2024-05-08T08:27:55.277362+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277362+0000","last_undegraded":"2024-05-08T08:27:55.277362+0000","last_fullsized":"2024-05-08T08:27:55.277362+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18586,"ondisk_log_size":18586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574902,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574902,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574902,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323351,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234027,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43148,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328692,"statfs":{"total":96636764160,"available":96592580608,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574902,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77520,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294320,"statfs":{"total":96636764160,"available":96557383680,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:29:09.259 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:09.259 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:09.260 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:09.260 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:29:29.260 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:29:29.575 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:29:29.576 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:29:29.587 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-05-08T08:29:28.475556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18705,"ondisk_log_size":18705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388308,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586412,"statfs":{"total":773094113280,"available":772696485888,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002571"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5528,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546903+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546903+0000","last_peered":"2024-05-08T08:27:53.546903+0000","last_clean":"2024-05-08T08:27:53.546903+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546903+0000","last_undegraded":"2024-05-08T08:27:53.546903+0000","last_fullsized":"2024-05-08T08:27:53.546903+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3792,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544431+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544431+0000","last_peered":"2024-05-08T08:27:53.544431+0000","last_clean":"2024-05-08T08:27:53.544431+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544431+0000","last_undegraded":"2024-05-08T08:27:53.544431+0000","last_fullsized":"2024-05-08T08:27:53.544431+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2159,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547711+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547711+0000","last_peered":"2024-05-08T08:27:53.547711+0000","last_clean":"2024-05-08T08:27:53.547711+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547711+0000","last_undegraded":"2024-05-08T08:27:53.547711+0000","last_fullsized":"2024-05-08T08:27:53.547711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7701,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547342+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547342+0000","last_peered":"2024-05-08T08:27:53.547342+0000","last_clean":"2024-05-08T08:27:53.547342+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547342+0000","last_undegraded":"2024-05-08T08:27:53.547342+0000","last_fullsized":"2024-05-08T08:27:53.547342+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3063,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550759+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.550759+0000","last_peered":"2024-05-08T08:27:53.550759+0000","last_clean":"2024-05-08T08:27:53.550759+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.550759+0000","last_undegraded":"2024-05-08T08:27:53.550759+0000","last_fullsized":"2024-05-08T08:27:53.550759+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12557,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634730+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634730+0000","last_peered":"2024-05-08T08:27:53.634730+0000","last_clean":"2024-05-08T08:27:53.634730+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634730+0000","last_undegraded":"2024-05-08T08:27:53.634730+0000","last_fullsized":"2024-05-08T08:27:53.634730+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547223+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547223+0000","last_peered":"2024-05-08T08:27:53.547223+0000","last_clean":"2024-05-08T08:27:53.547223+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547223+0000","last_undegraded":"2024-05-08T08:27:53.547223+0000","last_fullsized":"2024-05-08T08:27:53.547223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12805,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544766+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544766+0000","last_peered":"2024-05-08T08:27:53.544766+0000","last_clean":"2024-05-08T08:27:53.544766+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544766+0000","last_undegraded":"2024-05-08T08:27:53.544766+0000","last_fullsized":"2024-05-08T08:27:53.544766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551322+0000","last_change":"2024-05-08T08:27:53.551322+0000","last_active":"2024-05-08T08:27:53.551322+0000","last_peered":"2024-05-08T08:27:53.551322+0000","last_clean":"2024-05-08T08:27:53.551322+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:27:53.551322+0000","last_undegraded":"2024-05-08T08:27:53.551322+0000","last_fullsized":"2024-05-08T08:27:53.551322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:06.255241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2153,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547134+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547134+0000","last_peered":"2024-05-08T08:27:53.547134+0000","last_clean":"2024-05-08T08:27:53.547134+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547134+0000","last_undegraded":"2024-05-08T08:27:53.547134+0000","last_fullsized":"2024-05-08T08:27:53.547134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4195,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547443+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547443+0000","last_peered":"2024-05-08T08:27:53.547443+0000","last_clean":"2024-05-08T08:27:53.547443+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547443+0000","last_undegraded":"2024-05-08T08:27:53.547443+0000","last_fullsized":"2024-05-08T08:27:53.547443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550370+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.550370+0000","last_peered":"2024-05-08T08:27:53.550370+0000","last_clean":"2024-05-08T08:27:53.550370+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.550370+0000","last_undegraded":"2024-05-08T08:27:53.550370+0000","last_fullsized":"2024-05-08T08:27:53.550370+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546415+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546415+0000","last_peered":"2024-05-08T08:27:53.546415+0000","last_clean":"2024-05-08T08:27:53.546415+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546415+0000","last_undegraded":"2024-05-08T08:27:53.546415+0000","last_fullsized":"2024-05-08T08:27:53.546415+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9224,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547786+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547786+0000","last_peered":"2024-05-08T08:27:53.547786+0000","last_clean":"2024-05-08T08:27:53.547786+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547786+0000","last_undegraded":"2024-05-08T08:27:53.547786+0000","last_fullsized":"2024-05-08T08:27:53.547786+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546429+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546429+0000","last_peered":"2024-05-08T08:27:53.546429+0000","last_clean":"2024-05-08T08:27:53.546429+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546429+0000","last_undegraded":"2024-05-08T08:27:53.546429+0000","last_fullsized":"2024-05-08T08:27:53.546429+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5738,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547290+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547290+0000","last_peered":"2024-05-08T08:27:53.547290+0000","last_clean":"2024-05-08T08:27:53.547290+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547290+0000","last_undegraded":"2024-05-08T08:27:53.547290+0000","last_fullsized":"2024-05-08T08:27:53.547290+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":255,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634681+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634681+0000","last_peered":"2024-05-08T08:27:53.634681+0000","last_clean":"2024-05-08T08:27:53.634681+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634681+0000","last_undegraded":"2024-05-08T08:27:53.634681+0000","last_fullsized":"2024-05-08T08:27:53.634681+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7983,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544782+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544782+0000","last_peered":"2024-05-08T08:27:53.544782+0000","last_clean":"2024-05-08T08:27:53.544782+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544782+0000","last_undegraded":"2024-05-08T08:27:53.544782+0000","last_fullsized":"2024-05-08T08:27:53.544782+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544971+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544971+0000","last_peered":"2024-05-08T08:27:53.544971+0000","last_clean":"2024-05-08T08:27:53.544971+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544971+0000","last_undegraded":"2024-05-08T08:27:53.544971+0000","last_fullsized":"2024-05-08T08:27:53.544971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3342,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546856+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546856+0000","last_peered":"2024-05-08T08:27:53.546856+0000","last_clean":"2024-05-08T08:27:53.546856+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546856+0000","last_undegraded":"2024-05-08T08:27:53.546856+0000","last_fullsized":"2024-05-08T08:27:53.546856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3247,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546696+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546696+0000","last_peered":"2024-05-08T08:27:53.546696+0000","last_clean":"2024-05-08T08:27:53.546696+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546696+0000","last_undegraded":"2024-05-08T08:27:53.546696+0000","last_fullsized":"2024-05-08T08:27:53.546696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":474,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547156+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.547156+0000","last_peered":"2024-05-08T08:27:53.547156+0000","last_clean":"2024-05-08T08:27:53.547156+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.547156+0000","last_undegraded":"2024-05-08T08:27:53.547156+0000","last_fullsized":"2024-05-08T08:27:53.547156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547214+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547214+0000","last_peered":"2024-05-08T08:27:53.547214+0000","last_clean":"2024-05-08T08:27:53.547214+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547214+0000","last_undegraded":"2024-05-08T08:27:53.547214+0000","last_fullsized":"2024-05-08T08:27:53.547214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1932,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547281+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547281+0000","last_peered":"2024-05-08T08:27:53.547281+0000","last_clean":"2024-05-08T08:27:53.547281+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547281+0000","last_undegraded":"2024-05-08T08:27:53.547281+0000","last_fullsized":"2024-05-08T08:27:53.547281+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547087+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547087+0000","last_peered":"2024-05-08T08:27:53.547087+0000","last_clean":"2024-05-08T08:27:53.547087+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547087+0000","last_undegraded":"2024-05-08T08:27:53.547087+0000","last_fullsized":"2024-05-08T08:27:53.547087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544667+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544667+0000","last_peered":"2024-05-08T08:27:53.544667+0000","last_clean":"2024-05-08T08:27:53.544667+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544667+0000","last_undegraded":"2024-05-08T08:27:53.544667+0000","last_fullsized":"2024-05-08T08:27:53.544667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":161,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546949+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.546949+0000","last_peered":"2024-05-08T08:27:53.546949+0000","last_clean":"2024-05-08T08:27:53.546949+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.546949+0000","last_undegraded":"2024-05-08T08:27:53.546949+0000","last_fullsized":"2024-05-08T08:27:53.546949+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1989,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551373+0000","last_change":"2024-05-08T08:27:53.551373+0000","last_active":"2024-05-08T08:27:53.551373+0000","last_peered":"2024-05-08T08:27:53.551373+0000","last_clean":"2024-05-08T08:27:53.551373+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:27:53.551373+0000","last_undegraded":"2024-05-08T08:27:53.551373+0000","last_fullsized":"2024-05-08T08:27:53.551373+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:02.496828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546796+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546796+0000","last_peered":"2024-05-08T08:27:53.546796+0000","last_clean":"2024-05-08T08:27:53.546796+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546796+0000","last_undegraded":"2024-05-08T08:27:53.546796+0000","last_fullsized":"2024-05-08T08:27:53.546796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":5228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.553223+0000","last_change":"2024-05-08T08:27:53.553223+0000","last_active":"2024-05-08T08:27:53.553223+0000","last_peered":"2024-05-08T08:27:53.553223+0000","last_clean":"2024-05-08T08:27:53.553223+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:27:53.553223+0000","last_undegraded":"2024-05-08T08:27:53.553223+0000","last_fullsized":"2024-05-08T08:27:53.553223+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:36.911152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1619,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634643+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634643+0000","last_peered":"2024-05-08T08:27:53.634643+0000","last_clean":"2024-05-08T08:27:53.634643+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634643+0000","last_undegraded":"2024-05-08T08:27:53.634643+0000","last_fullsized":"2024-05-08T08:27:53.634643+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2016,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545099+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545099+0000","last_peered":"2024-05-08T08:27:53.545099+0000","last_clean":"2024-05-08T08:27:53.545099+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545099+0000","last_undegraded":"2024-05-08T08:27:53.545099+0000","last_fullsized":"2024-05-08T08:27:53.545099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2603,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544342+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544342+0000","last_peered":"2024-05-08T08:27:53.544342+0000","last_clean":"2024-05-08T08:27:53.544342+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544342+0000","last_undegraded":"2024-05-08T08:27:53.544342+0000","last_fullsized":"2024-05-08T08:27:53.544342+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4595,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547229+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547229+0000","last_peered":"2024-05-08T08:27:53.547229+0000","last_clean":"2024-05-08T08:27:53.547229+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547229+0000","last_undegraded":"2024-05-08T08:27:53.547229+0000","last_fullsized":"2024-05-08T08:27:53.547229+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1907,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547224+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547224+0000","last_peered":"2024-05-08T08:27:53.547224+0000","last_clean":"2024-05-08T08:27:53.547224+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547224+0000","last_undegraded":"2024-05-08T08:27:53.547224+0000","last_fullsized":"2024-05-08T08:27:53.547224+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5077,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285003+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285003+0000","last_peered":"2024-05-08T08:27:54.285003+0000","last_clean":"2024-05-08T08:27:54.285003+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285003+0000","last_undegraded":"2024-05-08T08:27:54.285003+0000","last_fullsized":"2024-05-08T08:27:54.285003+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1952,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547417+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547417+0000","last_peered":"2024-05-08T08:27:53.547417+0000","last_clean":"2024-05-08T08:27:53.547417+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547417+0000","last_undegraded":"2024-05-08T08:27:53.547417+0000","last_fullsized":"2024-05-08T08:27:53.547417+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3090,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546324+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546324+0000","last_peered":"2024-05-08T08:27:53.546324+0000","last_clean":"2024-05-08T08:27:53.546324+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546324+0000","last_undegraded":"2024-05-08T08:27:53.546324+0000","last_fullsized":"2024-05-08T08:27:53.546324+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1903,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547024+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547024+0000","last_peered":"2024-05-08T08:27:53.547024+0000","last_clean":"2024-05-08T08:27:53.547024+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547024+0000","last_undegraded":"2024-05-08T08:27:53.547024+0000","last_fullsized":"2024-05-08T08:27:53.547024+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546685+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546685+0000","last_peered":"2024-05-08T08:27:53.546685+0000","last_clean":"2024-05-08T08:27:53.546685+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546685+0000","last_undegraded":"2024-05-08T08:27:53.546685+0000","last_fullsized":"2024-05-08T08:27:53.546685+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4734,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277362+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277362+0000","last_peered":"2024-05-08T08:27:55.277362+0000","last_clean":"2024-05-08T08:27:55.277362+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277362+0000","last_undegraded":"2024-05-08T08:27:55.277362+0000","last_fullsized":"2024-05-08T08:27:55.277362+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18586,"ondisk_log_size":18586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323355,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234031,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43148,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328692,"statfs":{"total":96636764160,"available":96592580608,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77520,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294320,"statfs":{"total":96636764160,"available":96557383680,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:29:29.589 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:29.590 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:29.590 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:29.590 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:29:49.591 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:29:49.906 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:29:49.906 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:29:49.918 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2024-05-08T08:29:48.480221+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18705,"ondisk_log_size":18705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388308,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586412,"statfs":{"total":773094113280,"available":772696485888,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002905"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5528,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546903+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546903+0000","last_peered":"2024-05-08T08:27:53.546903+0000","last_clean":"2024-05-08T08:27:53.546903+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546903+0000","last_undegraded":"2024-05-08T08:27:53.546903+0000","last_fullsized":"2024-05-08T08:27:53.546903+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3793,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544491+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544491+0000","last_peered":"2024-05-08T08:27:53.544491+0000","last_clean":"2024-05-08T08:27:53.544491+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544491+0000","last_undegraded":"2024-05-08T08:27:53.544491+0000","last_fullsized":"2024-05-08T08:27:53.544491+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2159,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547711+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547711+0000","last_peered":"2024-05-08T08:27:53.547711+0000","last_clean":"2024-05-08T08:27:53.547711+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547711+0000","last_undegraded":"2024-05-08T08:27:53.547711+0000","last_fullsized":"2024-05-08T08:27:53.547711+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7701,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547342+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547342+0000","last_peered":"2024-05-08T08:27:53.547342+0000","last_clean":"2024-05-08T08:27:53.547342+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547342+0000","last_undegraded":"2024-05-08T08:27:53.547342+0000","last_fullsized":"2024-05-08T08:27:53.547342+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3063,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550759+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.550759+0000","last_peered":"2024-05-08T08:27:53.550759+0000","last_clean":"2024-05-08T08:27:53.550759+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.550759+0000","last_undegraded":"2024-05-08T08:27:53.550759+0000","last_fullsized":"2024-05-08T08:27:53.550759+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12557,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634730+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634730+0000","last_peered":"2024-05-08T08:27:53.634730+0000","last_clean":"2024-05-08T08:27:53.634730+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634730+0000","last_undegraded":"2024-05-08T08:27:53.634730+0000","last_fullsized":"2024-05-08T08:27:53.634730+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547223+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547223+0000","last_peered":"2024-05-08T08:27:53.547223+0000","last_clean":"2024-05-08T08:27:53.547223+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547223+0000","last_undegraded":"2024-05-08T08:27:53.547223+0000","last_fullsized":"2024-05-08T08:27:53.547223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12805,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544766+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544766+0000","last_peered":"2024-05-08T08:27:53.544766+0000","last_clean":"2024-05-08T08:27:53.544766+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544766+0000","last_undegraded":"2024-05-08T08:27:53.544766+0000","last_fullsized":"2024-05-08T08:27:53.544766+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551322+0000","last_change":"2024-05-08T08:27:53.551322+0000","last_active":"2024-05-08T08:27:53.551322+0000","last_peered":"2024-05-08T08:27:53.551322+0000","last_clean":"2024-05-08T08:27:53.551322+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:27:53.551322+0000","last_undegraded":"2024-05-08T08:27:53.551322+0000","last_fullsized":"2024-05-08T08:27:53.551322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:06.255241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2153,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547134+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547134+0000","last_peered":"2024-05-08T08:27:53.547134+0000","last_clean":"2024-05-08T08:27:53.547134+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547134+0000","last_undegraded":"2024-05-08T08:27:53.547134+0000","last_fullsized":"2024-05-08T08:27:53.547134+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547493+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547493+0000","last_peered":"2024-05-08T08:27:53.547493+0000","last_clean":"2024-05-08T08:27:53.547493+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547493+0000","last_undegraded":"2024-05-08T08:27:53.547493+0000","last_fullsized":"2024-05-08T08:27:53.547493+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550370+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.550370+0000","last_peered":"2024-05-08T08:27:53.550370+0000","last_clean":"2024-05-08T08:27:53.550370+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.550370+0000","last_undegraded":"2024-05-08T08:27:53.550370+0000","last_fullsized":"2024-05-08T08:27:53.550370+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546415+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546415+0000","last_peered":"2024-05-08T08:27:53.546415+0000","last_clean":"2024-05-08T08:27:53.546415+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546415+0000","last_undegraded":"2024-05-08T08:27:53.546415+0000","last_fullsized":"2024-05-08T08:27:53.546415+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9224,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547786+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547786+0000","last_peered":"2024-05-08T08:27:53.547786+0000","last_clean":"2024-05-08T08:27:53.547786+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547786+0000","last_undegraded":"2024-05-08T08:27:53.547786+0000","last_fullsized":"2024-05-08T08:27:53.547786+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546429+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546429+0000","last_peered":"2024-05-08T08:27:53.546429+0000","last_clean":"2024-05-08T08:27:53.546429+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546429+0000","last_undegraded":"2024-05-08T08:27:53.546429+0000","last_fullsized":"2024-05-08T08:27:53.546429+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5738,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547290+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547290+0000","last_peered":"2024-05-08T08:27:53.547290+0000","last_clean":"2024-05-08T08:27:53.547290+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547290+0000","last_undegraded":"2024-05-08T08:27:53.547290+0000","last_fullsized":"2024-05-08T08:27:53.547290+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":255,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634681+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634681+0000","last_peered":"2024-05-08T08:27:53.634681+0000","last_clean":"2024-05-08T08:27:53.634681+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634681+0000","last_undegraded":"2024-05-08T08:27:53.634681+0000","last_fullsized":"2024-05-08T08:27:53.634681+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7984,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544838+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544838+0000","last_peered":"2024-05-08T08:27:53.544838+0000","last_clean":"2024-05-08T08:27:53.544838+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544838+0000","last_undegraded":"2024-05-08T08:27:53.544838+0000","last_fullsized":"2024-05-08T08:27:53.544838+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544971+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544971+0000","last_peered":"2024-05-08T08:27:53.544971+0000","last_clean":"2024-05-08T08:27:53.544971+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544971+0000","last_undegraded":"2024-05-08T08:27:53.544971+0000","last_fullsized":"2024-05-08T08:27:53.544971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3342,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546856+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546856+0000","last_peered":"2024-05-08T08:27:53.546856+0000","last_clean":"2024-05-08T08:27:53.546856+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546856+0000","last_undegraded":"2024-05-08T08:27:53.546856+0000","last_fullsized":"2024-05-08T08:27:53.546856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3247,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546696+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546696+0000","last_peered":"2024-05-08T08:27:53.546696+0000","last_clean":"2024-05-08T08:27:53.546696+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546696+0000","last_undegraded":"2024-05-08T08:27:53.546696+0000","last_fullsized":"2024-05-08T08:27:53.546696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":474,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547156+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.547156+0000","last_peered":"2024-05-08T08:27:53.547156+0000","last_clean":"2024-05-08T08:27:53.547156+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.547156+0000","last_undegraded":"2024-05-08T08:27:53.547156+0000","last_fullsized":"2024-05-08T08:27:53.547156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":254,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547214+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547214+0000","last_peered":"2024-05-08T08:27:53.547214+0000","last_clean":"2024-05-08T08:27:53.547214+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547214+0000","last_undegraded":"2024-05-08T08:27:53.547214+0000","last_fullsized":"2024-05-08T08:27:53.547214+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1933,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547331+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547331+0000","last_peered":"2024-05-08T08:27:53.547331+0000","last_clean":"2024-05-08T08:27:53.547331+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547331+0000","last_undegraded":"2024-05-08T08:27:53.547331+0000","last_fullsized":"2024-05-08T08:27:53.547331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547087+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547087+0000","last_peered":"2024-05-08T08:27:53.547087+0000","last_clean":"2024-05-08T08:27:53.547087+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547087+0000","last_undegraded":"2024-05-08T08:27:53.547087+0000","last_fullsized":"2024-05-08T08:27:53.547087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544667+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544667+0000","last_peered":"2024-05-08T08:27:53.544667+0000","last_clean":"2024-05-08T08:27:53.544667+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544667+0000","last_undegraded":"2024-05-08T08:27:53.544667+0000","last_fullsized":"2024-05-08T08:27:53.544667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":162,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547025+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.547025+0000","last_peered":"2024-05-08T08:27:53.547025+0000","last_clean":"2024-05-08T08:27:53.547025+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.547025+0000","last_undegraded":"2024-05-08T08:27:53.547025+0000","last_fullsized":"2024-05-08T08:27:53.547025+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1989,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551373+0000","last_change":"2024-05-08T08:27:53.551373+0000","last_active":"2024-05-08T08:27:53.551373+0000","last_peered":"2024-05-08T08:27:53.551373+0000","last_clean":"2024-05-08T08:27:53.551373+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:27:53.551373+0000","last_undegraded":"2024-05-08T08:27:53.551373+0000","last_fullsized":"2024-05-08T08:27:53.551373+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:02.496828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546796+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546796+0000","last_peered":"2024-05-08T08:27:53.546796+0000","last_clean":"2024-05-08T08:27:53.546796+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546796+0000","last_undegraded":"2024-05-08T08:27:53.546796+0000","last_fullsized":"2024-05-08T08:27:53.546796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":5228,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.553223+0000","last_change":"2024-05-08T08:27:53.553223+0000","last_active":"2024-05-08T08:27:53.553223+0000","last_peered":"2024-05-08T08:27:53.553223+0000","last_clean":"2024-05-08T08:27:53.553223+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:27:53.553223+0000","last_undegraded":"2024-05-08T08:27:53.553223+0000","last_fullsized":"2024-05-08T08:27:53.553223+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:36.911152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1620,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634696+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634696+0000","last_peered":"2024-05-08T08:27:53.634696+0000","last_clean":"2024-05-08T08:27:53.634696+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634696+0000","last_undegraded":"2024-05-08T08:27:53.634696+0000","last_fullsized":"2024-05-08T08:27:53.634696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2016,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545099+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545099+0000","last_peered":"2024-05-08T08:27:53.545099+0000","last_clean":"2024-05-08T08:27:53.545099+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545099+0000","last_undegraded":"2024-05-08T08:27:53.545099+0000","last_fullsized":"2024-05-08T08:27:53.545099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2604,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544495+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544495+0000","last_peered":"2024-05-08T08:27:53.544495+0000","last_clean":"2024-05-08T08:27:53.544495+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544495+0000","last_undegraded":"2024-05-08T08:27:53.544495+0000","last_fullsized":"2024-05-08T08:27:53.544495+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4596,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547282+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547282+0000","last_peered":"2024-05-08T08:27:53.547282+0000","last_clean":"2024-05-08T08:27:53.547282+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547282+0000","last_undegraded":"2024-05-08T08:27:53.547282+0000","last_fullsized":"2024-05-08T08:27:53.547282+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1908,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547267+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547267+0000","last_peered":"2024-05-08T08:27:53.547267+0000","last_clean":"2024-05-08T08:27:53.547267+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547267+0000","last_undegraded":"2024-05-08T08:27:53.547267+0000","last_fullsized":"2024-05-08T08:27:53.547267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5078,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285076+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285076+0000","last_peered":"2024-05-08T08:27:54.285076+0000","last_clean":"2024-05-08T08:27:54.285076+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285076+0000","last_undegraded":"2024-05-08T08:27:54.285076+0000","last_fullsized":"2024-05-08T08:27:54.285076+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1953,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547462+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547462+0000","last_peered":"2024-05-08T08:27:53.547462+0000","last_clean":"2024-05-08T08:27:53.547462+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547462+0000","last_undegraded":"2024-05-08T08:27:53.547462+0000","last_fullsized":"2024-05-08T08:27:53.547462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3091,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546382+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546382+0000","last_peered":"2024-05-08T08:27:53.546382+0000","last_clean":"2024-05-08T08:27:53.546382+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546382+0000","last_undegraded":"2024-05-08T08:27:53.546382+0000","last_fullsized":"2024-05-08T08:27:53.546382+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1904,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547075+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547075+0000","last_peered":"2024-05-08T08:27:53.547075+0000","last_clean":"2024-05-08T08:27:53.547075+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547075+0000","last_undegraded":"2024-05-08T08:27:53.547075+0000","last_fullsized":"2024-05-08T08:27:53.547075+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3493,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546746+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546746+0000","last_peered":"2024-05-08T08:27:53.546746+0000","last_clean":"2024-05-08T08:27:53.546746+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546746+0000","last_undegraded":"2024-05-08T08:27:53.546746+0000","last_fullsized":"2024-05-08T08:27:53.546746+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4735,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277400+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277400+0000","last_peered":"2024-05-08T08:27:55.277400+0000","last_clean":"2024-05-08T08:27:55.277400+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277400+0000","last_undegraded":"2024-05-08T08:27:55.277400+0000","last_fullsized":"2024-05-08T08:27:55.277400+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18586,"ondisk_log_size":18586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574910,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323359,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574911,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234035,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43148,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328692,"statfs":{"total":96636764160,"available":96592580608,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77520,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294320,"statfs":{"total":96636764160,"available":96557383680,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:29:49.920 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:49.921 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:49.921 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:29:49.921 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:30:09.923 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:30:10.255 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:30:10.256 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:30:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":313,"stamp":"2024-05-08T08:30:08.484555+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18705,"ondisk_log_size":18705,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388308,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586412,"statfs":{"total":773094113280,"available":772696485888,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002774"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5528,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546903+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546903+0000","last_peered":"2024-05-08T08:27:53.546903+0000","last_clean":"2024-05-08T08:27:53.546903+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546903+0000","last_undegraded":"2024-05-08T08:27:53.546903+0000","last_fullsized":"2024-05-08T08:27:53.546903+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3793,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544491+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544491+0000","last_peered":"2024-05-08T08:27:53.544491+0000","last_clean":"2024-05-08T08:27:53.544491+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544491+0000","last_undegraded":"2024-05-08T08:27:53.544491+0000","last_fullsized":"2024-05-08T08:27:53.544491+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2160,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547762+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547762+0000","last_peered":"2024-05-08T08:27:53.547762+0000","last_clean":"2024-05-08T08:27:53.547762+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547762+0000","last_undegraded":"2024-05-08T08:27:53.547762+0000","last_fullsized":"2024-05-08T08:27:53.547762+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7702,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547401+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547401+0000","last_peered":"2024-05-08T08:27:53.547401+0000","last_clean":"2024-05-08T08:27:53.547401+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547401+0000","last_undegraded":"2024-05-08T08:27:53.547401+0000","last_fullsized":"2024-05-08T08:27:53.547401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3064,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.552253+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.552253+0000","last_peered":"2024-05-08T08:27:53.552253+0000","last_clean":"2024-05-08T08:27:53.552253+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.552253+0000","last_undegraded":"2024-05-08T08:27:53.552253+0000","last_fullsized":"2024-05-08T08:27:53.552253+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12558,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634773+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634773+0000","last_peered":"2024-05-08T08:27:53.634773+0000","last_clean":"2024-05-08T08:27:53.634773+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634773+0000","last_undegraded":"2024-05-08T08:27:53.634773+0000","last_fullsized":"2024-05-08T08:27:53.634773+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547223+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547223+0000","last_peered":"2024-05-08T08:27:53.547223+0000","last_clean":"2024-05-08T08:27:53.547223+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547223+0000","last_undegraded":"2024-05-08T08:27:53.547223+0000","last_fullsized":"2024-05-08T08:27:53.547223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12806,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544824+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544824+0000","last_peered":"2024-05-08T08:27:53.544824+0000","last_clean":"2024-05-08T08:27:53.544824+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544824+0000","last_undegraded":"2024-05-08T08:27:53.544824+0000","last_fullsized":"2024-05-08T08:27:53.544824+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"134'781","reported_seq":2492,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551322+0000","last_change":"2024-05-08T08:27:53.551322+0000","last_active":"2024-05-08T08:27:53.551322+0000","last_peered":"2024-05-08T08:27:53.551322+0000","last_clean":"2024-05-08T08:27:53.551322+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:27:53.551322+0000","last_undegraded":"2024-05-08T08:27:53.551322+0000","last_fullsized":"2024-05-08T08:27:53.551322+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":781,"log_dups_size":0,"ondisk_log_size":781,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:48:06.255241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2154,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547194+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547194+0000","last_peered":"2024-05-08T08:27:53.547194+0000","last_clean":"2024-05-08T08:27:53.547194+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547194+0000","last_undegraded":"2024-05-08T08:27:53.547194+0000","last_fullsized":"2024-05-08T08:27:53.547194+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547493+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547493+0000","last_peered":"2024-05-08T08:27:53.547493+0000","last_clean":"2024-05-08T08:27:53.547493+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547493+0000","last_undegraded":"2024-05-08T08:27:53.547493+0000","last_fullsized":"2024-05-08T08:27:53.547493+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1536,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.550370+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.550370+0000","last_peered":"2024-05-08T08:27:53.550370+0000","last_clean":"2024-05-08T08:27:53.550370+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.550370+0000","last_undegraded":"2024-05-08T08:27:53.550370+0000","last_fullsized":"2024-05-08T08:27:53.550370+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546415+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546415+0000","last_peered":"2024-05-08T08:27:53.546415+0000","last_clean":"2024-05-08T08:27:53.546415+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546415+0000","last_undegraded":"2024-05-08T08:27:53.546415+0000","last_fullsized":"2024-05-08T08:27:53.546415+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9224,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547786+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547786+0000","last_peered":"2024-05-08T08:27:53.547786+0000","last_clean":"2024-05-08T08:27:53.547786+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547786+0000","last_undegraded":"2024-05-08T08:27:53.547786+0000","last_fullsized":"2024-05-08T08:27:53.547786+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546429+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546429+0000","last_peered":"2024-05-08T08:27:53.546429+0000","last_clean":"2024-05-08T08:27:53.546429+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546429+0000","last_undegraded":"2024-05-08T08:27:53.546429+0000","last_fullsized":"2024-05-08T08:27:53.546429+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5739,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547344+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547344+0000","last_peered":"2024-05-08T08:27:53.547344+0000","last_clean":"2024-05-08T08:27:53.547344+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547344+0000","last_undegraded":"2024-05-08T08:27:53.547344+0000","last_fullsized":"2024-05-08T08:27:53.547344+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":255,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634681+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634681+0000","last_peered":"2024-05-08T08:27:53.634681+0000","last_clean":"2024-05-08T08:27:53.634681+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634681+0000","last_undegraded":"2024-05-08T08:27:53.634681+0000","last_fullsized":"2024-05-08T08:27:53.634681+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7984,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544838+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544838+0000","last_peered":"2024-05-08T08:27:53.544838+0000","last_clean":"2024-05-08T08:27:53.544838+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544838+0000","last_undegraded":"2024-05-08T08:27:53.544838+0000","last_fullsized":"2024-05-08T08:27:53.544838+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544971+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544971+0000","last_peered":"2024-05-08T08:27:53.544971+0000","last_clean":"2024-05-08T08:27:53.544971+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544971+0000","last_undegraded":"2024-05-08T08:27:53.544971+0000","last_fullsized":"2024-05-08T08:27:53.544971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3342,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546856+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546856+0000","last_peered":"2024-05-08T08:27:53.546856+0000","last_clean":"2024-05-08T08:27:53.546856+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546856+0000","last_undegraded":"2024-05-08T08:27:53.546856+0000","last_fullsized":"2024-05-08T08:27:53.546856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3247,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546696+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546696+0000","last_peered":"2024-05-08T08:27:53.546696+0000","last_clean":"2024-05-08T08:27:53.546696+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546696+0000","last_undegraded":"2024-05-08T08:27:53.546696+0000","last_fullsized":"2024-05-08T08:27:53.546696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":474,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547156+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.547156+0000","last_peered":"2024-05-08T08:27:53.547156+0000","last_clean":"2024-05-08T08:27:53.547156+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.547156+0000","last_undegraded":"2024-05-08T08:27:53.547156+0000","last_fullsized":"2024-05-08T08:27:53.547156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":255,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547262+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547262+0000","last_peered":"2024-05-08T08:27:53.547262+0000","last_clean":"2024-05-08T08:27:53.547262+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547262+0000","last_undegraded":"2024-05-08T08:27:53.547262+0000","last_fullsized":"2024-05-08T08:27:53.547262+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1933,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547331+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547331+0000","last_peered":"2024-05-08T08:27:53.547331+0000","last_clean":"2024-05-08T08:27:53.547331+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547331+0000","last_undegraded":"2024-05-08T08:27:53.547331+0000","last_fullsized":"2024-05-08T08:27:53.547331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547087+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547087+0000","last_peered":"2024-05-08T08:27:53.547087+0000","last_clean":"2024-05-08T08:27:53.547087+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547087+0000","last_undegraded":"2024-05-08T08:27:53.547087+0000","last_fullsized":"2024-05-08T08:27:53.547087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544667+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544667+0000","last_peered":"2024-05-08T08:27:53.544667+0000","last_clean":"2024-05-08T08:27:53.544667+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544667+0000","last_undegraded":"2024-05-08T08:27:53.544667+0000","last_fullsized":"2024-05-08T08:27:53.544667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":162,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547025+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.547025+0000","last_peered":"2024-05-08T08:27:53.547025+0000","last_clean":"2024-05-08T08:27:53.547025+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.547025+0000","last_undegraded":"2024-05-08T08:27:53.547025+0000","last_fullsized":"2024-05-08T08:27:53.547025+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"136'324","reported_seq":1990,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.552009+0000","last_change":"2024-05-08T08:27:53.551373+0000","last_active":"2024-05-08T08:27:53.552009+0000","last_peered":"2024-05-08T08:27:53.552009+0000","last_clean":"2024-05-08T08:27:53.552009+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:27:53.552009+0000","last_undegraded":"2024-05-08T08:27:53.552009+0000","last_fullsized":"2024-05-08T08:27:53.552009+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":324,"log_dups_size":0,"ondisk_log_size":324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:02.496828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546796+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546796+0000","last_peered":"2024-05-08T08:27:53.546796+0000","last_clean":"2024-05-08T08:27:53.546796+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546796+0000","last_undegraded":"2024-05-08T08:27:53.546796+0000","last_fullsized":"2024-05-08T08:27:53.546796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"137'456","reported_seq":5229,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.553776+0000","last_change":"2024-05-08T08:27:53.553223+0000","last_active":"2024-05-08T08:27:53.553776+0000","last_peered":"2024-05-08T08:27:53.553776+0000","last_clean":"2024-05-08T08:27:53.553776+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:27:53.553776+0000","last_undegraded":"2024-05-08T08:27:53.553776+0000","last_fullsized":"2024-05-08T08:27:53.553776+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:25:03.585177+0000","last_clean_scrub_stamp":"2024-05-08T08:25:03.585177+0000","objects_scrubbed":0,"log_size":456,"log_dups_size":0,"ondisk_log_size":456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:54:36.911152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1620,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634696+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634696+0000","last_peered":"2024-05-08T08:27:53.634696+0000","last_clean":"2024-05-08T08:27:53.634696+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634696+0000","last_undegraded":"2024-05-08T08:27:53.634696+0000","last_fullsized":"2024-05-08T08:27:53.634696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2016,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545099+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545099+0000","last_peered":"2024-05-08T08:27:53.545099+0000","last_clean":"2024-05-08T08:27:53.545099+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545099+0000","last_undegraded":"2024-05-08T08:27:53.545099+0000","last_fullsized":"2024-05-08T08:27:53.545099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2604,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544495+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544495+0000","last_peered":"2024-05-08T08:27:53.544495+0000","last_clean":"2024-05-08T08:27:53.544495+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544495+0000","last_undegraded":"2024-05-08T08:27:53.544495+0000","last_fullsized":"2024-05-08T08:27:53.544495+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4596,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547282+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547282+0000","last_peered":"2024-05-08T08:27:53.547282+0000","last_clean":"2024-05-08T08:27:53.547282+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547282+0000","last_undegraded":"2024-05-08T08:27:53.547282+0000","last_fullsized":"2024-05-08T08:27:53.547282+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1908,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547267+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547267+0000","last_peered":"2024-05-08T08:27:53.547267+0000","last_clean":"2024-05-08T08:27:53.547267+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547267+0000","last_undegraded":"2024-05-08T08:27:53.547267+0000","last_fullsized":"2024-05-08T08:27:53.547267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5078,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285076+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285076+0000","last_peered":"2024-05-08T08:27:54.285076+0000","last_clean":"2024-05-08T08:27:54.285076+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285076+0000","last_undegraded":"2024-05-08T08:27:54.285076+0000","last_fullsized":"2024-05-08T08:27:54.285076+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1953,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547462+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547462+0000","last_peered":"2024-05-08T08:27:53.547462+0000","last_clean":"2024-05-08T08:27:53.547462+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547462+0000","last_undegraded":"2024-05-08T08:27:53.547462+0000","last_fullsized":"2024-05-08T08:27:53.547462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3091,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546382+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546382+0000","last_peered":"2024-05-08T08:27:53.546382+0000","last_clean":"2024-05-08T08:27:53.546382+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546382+0000","last_undegraded":"2024-05-08T08:27:53.546382+0000","last_fullsized":"2024-05-08T08:27:53.546382+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1904,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547075+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547075+0000","last_peered":"2024-05-08T08:27:53.547075+0000","last_clean":"2024-05-08T08:27:53.547075+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547075+0000","last_undegraded":"2024-05-08T08:27:53.547075+0000","last_fullsized":"2024-05-08T08:27:53.547075+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3493,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546746+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546746+0000","last_peered":"2024-05-08T08:27:53.546746+0000","last_clean":"2024-05-08T08:27:53.546746+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546746+0000","last_undegraded":"2024-05-08T08:27:53.546746+0000","last_fullsized":"2024-05-08T08:27:53.546746+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4735,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277400+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277400+0000","last_peered":"2024-05-08T08:27:55.277400+0000","last_clean":"2024-05-08T08:27:55.277400+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277400+0000","last_undegraded":"2024-05-08T08:27:55.277400+0000","last_fullsized":"2024-05-08T08:27:55.277400+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18586,"ondisk_log_size":18586,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55312,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316528,"statfs":{"total":96636764160,"available":96580124672,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323363,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55304,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316536,"statfs":{"total":96636764160,"available":96580132864,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234039,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43148,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328692,"statfs":{"total":96636764160,"available":96592580608,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77520,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294320,"statfs":{"total":96636764160,"available":96557383680,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:30:10.269 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:30:10.270 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:30:10.270 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-05-08T08:25:03.585177+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=25, tm_sec=3, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=8, tm_min=27, tm_sec=42, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T08:30:10.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.17 2024-05-08T08:30:10.567 INFO:teuthology.orchestra.run.smithi031.stderr:instructing pg 3.17 on osd.0 to deep-scrub 2024-05-08T08:30:10.579 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.2 2024-05-08T08:30:10.876 INFO:teuthology.orchestra.run.smithi031.stderr:instructing pg 3.2 on osd.3 to deep-scrub 2024-05-08T08:30:10.889 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg deep-scrub 3.8 2024-05-08T08:30:11.185 INFO:teuthology.orchestra.run.smithi031.stderr:instructing pg 3.8 on osd.3 to deep-scrub 2024-05-08T08:30:11.198 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T08:30:31.199 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T08:30:31.515 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-05-08T08:30:31.515 INFO:teuthology.orchestra.run.smithi031.stderr:dumped all 2024-05-08T08:30:31.526 INFO:teuthology.orchestra.run.smithi031.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-05-08T08:30:30.489928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":279,"num_object_clones":0,"num_object_copies":558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":279,"num_whiteouts":273,"num_read":158395,"num_read_kb":861706,"num_write":22005,"num_write_kb":474524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18737,"ondisk_log_size":18737,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":388268,"kb_used_data":15448,"kb_used_omap":55,"kb_used_meta":372680,"kb_avail":754586452,"statfs":{"total":773094113280,"available":772696526848,"internally_reserved":0,"allocated":15818752,"data_stored":8821894,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":56817,"internal_metadata":381624847},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002976"},"pg_stats":[{"pgid":"3.1f","version":"158'750","reported_seq":5528,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546903+0000","last_change":"2024-05-08T08:27:47.006866+0000","last_active":"2024-05-08T08:27:53.546903+0000","last_peered":"2024-05-08T08:27:53.546903+0000","last_clean":"2024-05-08T08:27:53.546903+0000","last_became_active":"2024-05-08T08:27:02.044401+0000","last_became_peered":"2024-05-08T08:27:02.044401+0000","last_unstale":"2024-05-08T08:27:53.546903+0000","last_undegraded":"2024-05-08T08:27:53.546903+0000","last_fullsized":"2024-05-08T08:27:53.546903+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'750","last_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_deep_scrub":"158'750","last_deep_scrub_stamp":"2024-05-08T08:27:47.006766+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.006766+0000","objects_scrubbed":10,"log_size":750,"log_dups_size":0,"ondisk_log_size":750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:20:17.538492+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0014708620000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":8492,"num_read_kb":28712,"num_write":727,"num_write_kb":7027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":5518517,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"158'807","reported_seq":3793,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544491+0000","last_change":"2024-05-08T08:27:47.237214+0000","last_active":"2024-05-08T08:27:53.544491+0000","last_peered":"2024-05-08T08:27:53.544491+0000","last_clean":"2024-05-08T08:27:53.544491+0000","last_became_active":"2024-05-08T08:27:28.176420+0000","last_became_peered":"2024-05-08T08:27:28.176420+0000","last_unstale":"2024-05-08T08:27:53.544491+0000","last_undegraded":"2024-05-08T08:27:53.544491+0000","last_fullsized":"2024-05-08T08:27:53.544491+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'807","last_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_deep_scrub":"158'807","last_deep_scrub_stamp":"2024-05-08T08:27:47.237063+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.237063+0000","objects_scrubbed":11,"log_size":807,"log_dups_size":0,"ondisk_log_size":807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:40.579341+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.00040041000000000002,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":5279,"num_read_kb":27120,"num_write":1196,"num_write_kb":17973,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":13235542,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"160'566","reported_seq":2160,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547762+0000","last_change":"2024-05-08T08:27:51.313998+0000","last_active":"2024-05-08T08:27:53.547762+0000","last_peered":"2024-05-08T08:27:53.547762+0000","last_clean":"2024-05-08T08:27:53.547762+0000","last_became_active":"2024-05-08T08:27:29.998506+0000","last_became_peered":"2024-05-08T08:27:29.998506+0000","last_unstale":"2024-05-08T08:27:53.547762+0000","last_undegraded":"2024-05-08T08:27:53.547762+0000","last_fullsized":"2024-05-08T08:27:53.547762+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'566","last_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_deep_scrub":"160'566","last_deep_scrub_stamp":"2024-05-08T08:27:51.313876+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.313876+0000","objects_scrubbed":4,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:27:19.281241+0000","scrub_duration":5,"objects_trimmed":14,"snaptrim_duration":0.057472337999999998,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":4,"num_read":2878,"num_read_kb":22100,"num_write":708,"num_write_kb":14054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1c","version":"161'802","reported_seq":7702,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547401+0000","last_change":"2024-05-08T08:27:52.289885+0000","last_active":"2024-05-08T08:27:53.547401+0000","last_peered":"2024-05-08T08:27:53.547401+0000","last_clean":"2024-05-08T08:27:53.547401+0000","last_became_active":"2024-05-08T08:27:29.998511+0000","last_became_peered":"2024-05-08T08:27:29.998511+0000","last_unstale":"2024-05-08T08:27:53.547401+0000","last_undegraded":"2024-05-08T08:27:53.547401+0000","last_fullsized":"2024-05-08T08:27:53.547401+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'802","last_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_deep_scrub":"161'802","last_deep_scrub_stamp":"2024-05-08T08:27:52.289743+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.289743+0000","objects_scrubbed":8,"log_size":802,"log_dups_size":0,"ondisk_log_size":802,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:11:22.030412+0000","scrub_duration":7,"objects_trimmed":12,"snaptrim_duration":0.031625630000000002,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":9772,"num_read_kb":74714,"num_write":999,"num_write_kb":23733,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"158'1113","reported_seq":3064,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.552253+0000","last_change":"2024-05-08T08:27:53.550759+0000","last_active":"2024-05-08T08:27:53.552253+0000","last_peered":"2024-05-08T08:27:53.552253+0000","last_clean":"2024-05-08T08:27:53.552253+0000","last_became_active":"2024-05-08T08:27:53.548452+0000","last_became_peered":"2024-05-08T08:27:53.548452+0000","last_unstale":"2024-05-08T08:27:53.552253+0000","last_undegraded":"2024-05-08T08:27:53.552253+0000","last_fullsized":"2024-05-08T08:27:53.552253+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1113","last_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_deep_scrub":"158'1113","last_deep_scrub_stamp":"2024-05-08T08:27:47.980925+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.980925+0000","objects_scrubbed":13,"log_size":1113,"log_dups_size":0,"ondisk_log_size":1113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:02:37.796212+0000","scrub_duration":13,"objects_trimmed":16,"snaptrim_duration":0.059238654000000002,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":6190,"num_read_kb":12620,"num_write":1705,"num_write_kb":13439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"158'476","reported_seq":12558,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634773+0000","last_change":"2024-05-08T08:27:48.574232+0000","last_active":"2024-05-08T08:27:53.634773+0000","last_peered":"2024-05-08T08:27:53.634773+0000","last_clean":"2024-05-08T08:27:53.634773+0000","last_became_active":"2024-05-08T08:27:23.146069+0000","last_became_peered":"2024-05-08T08:27:23.146069+0000","last_unstale":"2024-05-08T08:27:53.634773+0000","last_undegraded":"2024-05-08T08:27:53.634773+0000","last_fullsized":"2024-05-08T08:27:53.634773+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'476","last_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_deep_scrub":"158'476","last_deep_scrub_stamp":"2024-05-08T08:27:48.574109+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.574109+0000","objects_scrubbed":11,"log_size":476,"log_dups_size":0,"ondisk_log_size":476,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:57:40.427502+0000","scrub_duration":8,"objects_trimmed":12,"snaptrim_duration":0.030608232999999999,"stat_sum":{"num_bytes":19,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":10,"num_read":12132,"num_read_kb":37171,"num_write":501,"num_write_kb":15904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":57,"num_bytes_recovered":4006776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.19","version":"158'439","reported_seq":2375,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547223+0000","last_change":"2024-05-08T08:27:45.142058+0000","last_active":"2024-05-08T08:27:53.547223+0000","last_peered":"2024-05-08T08:27:53.547223+0000","last_clean":"2024-05-08T08:27:53.547223+0000","last_became_active":"2024-05-08T08:25:48.158408+0000","last_became_peered":"2024-05-08T08:25:48.158408+0000","last_unstale":"2024-05-08T08:27:53.547223+0000","last_undegraded":"2024-05-08T08:27:53.547223+0000","last_fullsized":"2024-05-08T08:27:53.547223+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'439","last_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_deep_scrub":"158'439","last_deep_scrub_stamp":"2024-05-08T08:27:45.141946+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.141946+0000","objects_scrubbed":6,"log_size":439,"log_dups_size":0,"ondisk_log_size":439,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:45:03.574915+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.0016328580000000001,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":2521,"num_read_kb":23485,"num_write":468,"num_write_kb":21322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1685084,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"158'415","reported_seq":12806,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544824+0000","last_change":"2024-05-08T08:27:48.274976+0000","last_active":"2024-05-08T08:27:53.544824+0000","last_peered":"2024-05-08T08:27:53.544824+0000","last_clean":"2024-05-08T08:27:53.544824+0000","last_became_active":"2024-05-08T08:27:28.177907+0000","last_became_peered":"2024-05-08T08:27:28.177907+0000","last_unstale":"2024-05-08T08:27:53.544824+0000","last_undegraded":"2024-05-08T08:27:53.544824+0000","last_fullsized":"2024-05-08T08:27:53.544824+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'415","last_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_deep_scrub":"158'415","last_deep_scrub_stamp":"2024-05-08T08:27:48.274866+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.274866+0000","objects_scrubbed":6,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:02:31.139113+0000","scrub_duration":6,"objects_trimmed":2,"snaptrim_duration":0.001479204,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":12647,"num_read_kb":31472,"num_write":393,"num_write_kb":16484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"163'790","reported_seq":2511,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:30:11.611124+0000","last_change":"2024-05-08T08:30:11.611124+0000","last_active":"2024-05-08T08:30:11.611124+0000","last_peered":"2024-05-08T08:30:11.611124+0000","last_clean":"2024-05-08T08:30:11.611124+0000","last_became_active":"2024-05-08T08:27:53.549323+0000","last_became_peered":"2024-05-08T08:27:53.549323+0000","last_unstale":"2024-05-08T08:30:11.611124+0000","last_undegraded":"2024-05-08T08:30:11.611124+0000","last_fullsized":"2024-05-08T08:30:11.611124+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'790","last_scrub_stamp":"2024-05-08T08:30:11.610961+0000","last_deep_scrub":"163'790","last_deep_scrub_stamp":"2024-05-08T08:30:11.610961+0000","last_clean_scrub_stamp":"2024-05-08T08:30:11.610961+0000","objects_scrubbed":9,"log_size":790,"log_dups_size":0,"ondisk_log_size":790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:31:25.500118+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.0023066509999999998,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":4241,"num_read_kb":7894,"num_write":1072,"num_write_kb":8249,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":404,"num_bytes_recovered":1964404,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"158'400","reported_seq":2154,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547194+0000","last_change":"2024-05-08T08:27:48.403762+0000","last_active":"2024-05-08T08:27:53.547194+0000","last_peered":"2024-05-08T08:27:53.547194+0000","last_clean":"2024-05-08T08:27:53.547194+0000","last_became_active":"2024-05-08T08:27:29.995378+0000","last_became_peered":"2024-05-08T08:27:29.995378+0000","last_unstale":"2024-05-08T08:27:53.547194+0000","last_undegraded":"2024-05-08T08:27:53.547194+0000","last_fullsized":"2024-05-08T08:27:53.547194+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'400","last_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_deep_scrub":"158'400","last_deep_scrub_stamp":"2024-05-08T08:27:48.403636+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.403636+0000","objects_scrubbed":10,"log_size":400,"log_dups_size":0,"ondisk_log_size":400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:09:42.085706+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.012364974000000001,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":2378,"num_read_kb":14614,"num_write":362,"num_write_kb":11778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"158'431","reported_seq":4196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547493+0000","last_change":"2024-05-08T08:27:48.091522+0000","last_active":"2024-05-08T08:27:53.547493+0000","last_peered":"2024-05-08T08:27:53.547493+0000","last_clean":"2024-05-08T08:27:53.547493+0000","last_became_active":"2024-05-08T08:27:14.989668+0000","last_became_peered":"2024-05-08T08:27:14.989668+0000","last_unstale":"2024-05-08T08:27:53.547493+0000","last_undegraded":"2024-05-08T08:27:53.547493+0000","last_fullsized":"2024-05-08T08:27:53.547493+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:48.091412+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.091412+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:31:14.862236+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.0035006619999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":4236,"num_read_kb":18440,"num_write":426,"num_write_kb":5782,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":336,"num_bytes_recovered":2396008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"158'515","reported_seq":1537,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.551127+0000","last_change":"2024-05-08T08:27:53.550370+0000","last_active":"2024-05-08T08:27:53.551127+0000","last_peered":"2024-05-08T08:27:53.551127+0000","last_clean":"2024-05-08T08:27:53.551127+0000","last_became_active":"2024-05-08T08:27:53.548659+0000","last_became_peered":"2024-05-08T08:27:53.548659+0000","last_unstale":"2024-05-08T08:27:53.551127+0000","last_undegraded":"2024-05-08T08:27:53.551127+0000","last_fullsized":"2024-05-08T08:27:53.551127+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'515","last_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_deep_scrub":"158'515","last_deep_scrub_stamp":"2024-05-08T08:27:47.594051+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.594051+0000","objects_scrubbed":6,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:42.090442+0000","scrub_duration":6,"objects_trimmed":20,"snaptrim_duration":0.053376155000000002,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1588,"num_read_kb":8808,"num_write":652,"num_write_kb":10691,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":123,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546415+0000","last_change":"2024-05-08T08:27:48.163728+0000","last_active":"2024-05-08T08:27:53.546415+0000","last_peered":"2024-05-08T08:27:53.546415+0000","last_clean":"2024-05-08T08:27:53.546415+0000","last_became_active":"2024-05-08T08:27:05.057126+0000","last_became_peered":"2024-05-08T08:27:05.057126+0000","last_unstale":"2024-05-08T08:27:53.546415+0000","last_undegraded":"2024-05-08T08:27:53.546415+0000","last_fullsized":"2024-05-08T08:27:53.546415+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.163601+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.163601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:47:19.001335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"158'431","reported_seq":9224,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547786+0000","last_change":"2024-05-08T08:27:49.363976+0000","last_active":"2024-05-08T08:27:53.547786+0000","last_peered":"2024-05-08T08:27:53.547786+0000","last_clean":"2024-05-08T08:27:53.547786+0000","last_became_active":"2024-05-08T08:27:29.995408+0000","last_became_peered":"2024-05-08T08:27:29.995408+0000","last_unstale":"2024-05-08T08:27:53.547786+0000","last_undegraded":"2024-05-08T08:27:53.547786+0000","last_fullsized":"2024-05-08T08:27:53.547786+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'431","last_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_deep_scrub":"158'431","last_deep_scrub_stamp":"2024-05-08T08:27:49.363818+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.363818+0000","objects_scrubbed":7,"log_size":431,"log_dups_size":0,"ondisk_log_size":431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:56:23.260598+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.001914509,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":8946,"num_read_kb":19929,"num_write":386,"num_write_kb":10132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546429+0000","last_change":"2024-05-08T08:27:49.129811+0000","last_active":"2024-05-08T08:27:53.546429+0000","last_peered":"2024-05-08T08:27:53.546429+0000","last_clean":"2024-05-08T08:27:53.546429+0000","last_became_active":"2024-05-08T08:27:05.060742+0000","last_became_peered":"2024-05-08T08:27:05.060742+0000","last_unstale":"2024-05-08T08:27:53.546429+0000","last_undegraded":"2024-05-08T08:27:53.546429+0000","last_fullsized":"2024-05-08T08:27:53.546429+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.129622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.129622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:24:30.381548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"158'459","reported_seq":5739,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547344+0000","last_change":"2024-05-08T08:27:49.064273+0000","last_active":"2024-05-08T08:27:53.547344+0000","last_peered":"2024-05-08T08:27:53.547344+0000","last_clean":"2024-05-08T08:27:53.547344+0000","last_became_active":"2024-05-08T08:27:28.177835+0000","last_became_peered":"2024-05-08T08:27:28.177835+0000","last_unstale":"2024-05-08T08:27:53.547344+0000","last_undegraded":"2024-05-08T08:27:53.547344+0000","last_fullsized":"2024-05-08T08:27:53.547344+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'459","last_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_deep_scrub":"158'459","last_deep_scrub_stamp":"2024-05-08T08:27:49.064138+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.064138+0000","objects_scrubbed":7,"log_size":459,"log_dups_size":0,"ondisk_log_size":459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:58:57.051661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0025127159999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5760,"num_read_kb":36923,"num_write":486,"num_write_kb":6645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":254,"num_bytes_recovered":6116510,"num_keys_recovered":22,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":255,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634681+0000","last_change":"2024-05-08T08:27:49.579063+0000","last_active":"2024-05-08T08:27:53.634681+0000","last_peered":"2024-05-08T08:27:53.634681+0000","last_clean":"2024-05-08T08:27:53.634681+0000","last_became_active":"2024-05-08T08:27:25.159931+0000","last_became_peered":"2024-05-08T08:27:25.159931+0000","last_unstale":"2024-05-08T08:27:53.634681+0000","last_undegraded":"2024-05-08T08:27:53.634681+0000","last_fullsized":"2024-05-08T08:27:53.634681+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.578942+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.578942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:04:02.977568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"159'858","reported_seq":7984,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544838+0000","last_change":"2024-05-08T08:27:50.288653+0000","last_active":"2024-05-08T08:27:53.544838+0000","last_peered":"2024-05-08T08:27:53.544838+0000","last_clean":"2024-05-08T08:27:53.544838+0000","last_became_active":"2024-05-08T08:27:28.180418+0000","last_became_peered":"2024-05-08T08:27:28.180418+0000","last_unstale":"2024-05-08T08:27:53.544838+0000","last_undegraded":"2024-05-08T08:27:53.544838+0000","last_fullsized":"2024-05-08T08:27:53.544838+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'858","last_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_deep_scrub":"159'858","last_deep_scrub_stamp":"2024-05-08T08:27:50.288518+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.288518+0000","objects_scrubbed":8,"log_size":858,"log_dups_size":0,"ondisk_log_size":858,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:49:02.118977+0000","scrub_duration":10,"objects_trimmed":6,"snaptrim_duration":0.017835917999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":10644,"num_read_kb":75533,"num_write":1074,"num_write_kb":39610,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":316,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544971+0000","last_change":"2024-05-08T08:27:46.224854+0000","last_active":"2024-05-08T08:27:53.544971+0000","last_peered":"2024-05-08T08:27:53.544971+0000","last_clean":"2024-05-08T08:27:53.544971+0000","last_became_active":"2024-05-08T08:24:56.566020+0000","last_became_peered":"2024-05-08T08:24:56.566020+0000","last_unstale":"2024-05-08T08:27:53.544971+0000","last_undegraded":"2024-05-08T08:27:53.544971+0000","last_fullsized":"2024-05-08T08:27:53.544971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:46.224724+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.224724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:55.230453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"158'525","reported_seq":3342,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546856+0000","last_change":"2024-05-08T08:27:45.084401+0000","last_active":"2024-05-08T08:27:53.546856+0000","last_peered":"2024-05-08T08:27:53.546856+0000","last_clean":"2024-05-08T08:27:53.546856+0000","last_became_active":"2024-05-08T08:25:48.158574+0000","last_became_peered":"2024-05-08T08:25:48.158574+0000","last_unstale":"2024-05-08T08:27:53.546856+0000","last_undegraded":"2024-05-08T08:27:53.546856+0000","last_fullsized":"2024-05-08T08:27:53.546856+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'525","last_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_deep_scrub":"158'525","last_deep_scrub_stamp":"2024-05-08T08:27:45.084286+0000","last_clean_scrub_stamp":"2024-05-08T08:27:45.084286+0000","objects_scrubbed":12,"log_size":525,"log_dups_size":0,"ondisk_log_size":525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:07.928639+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.001475638,"stat_sum":{"num_bytes":0,"num_objects":12,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":12,"num_read":3762,"num_read_kb":40058,"num_write":465,"num_write_kb":20014,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35,"num_bytes_recovered":1441963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"158'1277","reported_seq":3247,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546696+0000","last_change":"2024-05-08T08:27:48.969041+0000","last_active":"2024-05-08T08:27:53.546696+0000","last_peered":"2024-05-08T08:27:53.546696+0000","last_clean":"2024-05-08T08:27:53.546696+0000","last_became_active":"2024-05-08T08:27:23.152787+0000","last_became_peered":"2024-05-08T08:27:23.152787+0000","last_unstale":"2024-05-08T08:27:53.546696+0000","last_undegraded":"2024-05-08T08:27:53.546696+0000","last_fullsized":"2024-05-08T08:27:53.546696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'1277","last_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_deep_scrub":"158'1277","last_deep_scrub_stamp":"2024-05-08T08:27:48.968829+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.968829+0000","objects_scrubbed":13,"log_size":1277,"log_dups_size":0,"ondisk_log_size":1277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:30:47.962388+0000","scrub_duration":12,"objects_trimmed":18,"snaptrim_duration":0.033453995,"stat_sum":{"num_bytes":0,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":13,"num_read":7172,"num_read_kb":16501,"num_write":1894,"num_write_kb":9329,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"160'119","reported_seq":474,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547156+0000","last_change":"2024-05-08T08:27:51.301921+0000","last_active":"2024-05-08T08:27:53.547156+0000","last_peered":"2024-05-08T08:27:53.547156+0000","last_clean":"2024-05-08T08:27:53.547156+0000","last_became_active":"2024-05-08T08:25:45.140329+0000","last_became_peered":"2024-05-08T08:25:45.140329+0000","last_unstale":"2024-05-08T08:27:53.547156+0000","last_undegraded":"2024-05-08T08:27:53.547156+0000","last_fullsized":"2024-05-08T08:27:53.547156+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'119","last_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_deep_scrub":"160'119","last_deep_scrub_stamp":"2024-05-08T08:27:51.301779+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.301779+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:00:35.050219+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":255,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547262+0000","last_change":"2024-05-08T08:27:47.113226+0000","last_active":"2024-05-08T08:27:53.547262+0000","last_peered":"2024-05-08T08:27:53.547262+0000","last_clean":"2024-05-08T08:27:53.547262+0000","last_became_active":"2024-05-08T08:27:05.059295+0000","last_became_peered":"2024-05-08T08:27:05.059295+0000","last_unstale":"2024-05-08T08:27:53.547262+0000","last_undegraded":"2024-05-08T08:27:53.547262+0000","last_fullsized":"2024-05-08T08:27:53.547262+0000","mapping_epoch":128,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":129,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.113131+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.113131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:58:49.673624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"158'374","reported_seq":1933,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547331+0000","last_change":"2024-05-08T08:27:46.150183+0000","last_active":"2024-05-08T08:27:53.547331+0000","last_peered":"2024-05-08T08:27:53.547331+0000","last_clean":"2024-05-08T08:27:53.547331+0000","last_became_active":"2024-05-08T08:25:48.158894+0000","last_became_peered":"2024-05-08T08:25:48.158894+0000","last_unstale":"2024-05-08T08:27:53.547331+0000","last_undegraded":"2024-05-08T08:27:53.547331+0000","last_fullsized":"2024-05-08T08:27:53.547331+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'374","last_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_deep_scrub":"158'374","last_deep_scrub_stamp":"2024-05-08T08:27:46.150030+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.150030+0000","objects_scrubbed":11,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:51:15.521983+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.001180221,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":1055,"num_read_kb":15372,"num_write":376,"num_write_kb":11975,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43,"num_bytes_recovered":2850271,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547087+0000","last_change":"2024-05-08T08:27:48.348379+0000","last_active":"2024-05-08T08:27:53.547087+0000","last_peered":"2024-05-08T08:27:53.547087+0000","last_clean":"2024-05-08T08:27:53.547087+0000","last_became_active":"2024-05-08T08:24:56.680001+0000","last_became_peered":"2024-05-08T08:24:56.680001+0000","last_unstale":"2024-05-08T08:27:53.547087+0000","last_undegraded":"2024-05-08T08:27:53.547087+0000","last_fullsized":"2024-05-08T08:27:53.547087+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:48.348261+0000","last_clean_scrub_stamp":"2024-05-08T08:27:48.348261+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:47.871745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"158'805","reported_seq":6196,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544667+0000","last_change":"2024-05-08T08:27:46.131556+0000","last_active":"2024-05-08T08:27:53.544667+0000","last_peered":"2024-05-08T08:27:53.544667+0000","last_clean":"2024-05-08T08:27:53.544667+0000","last_became_active":"2024-05-08T08:26:44.069827+0000","last_became_peered":"2024-05-08T08:26:44.069827+0000","last_unstale":"2024-05-08T08:27:53.544667+0000","last_undegraded":"2024-05-08T08:27:53.544667+0000","last_fullsized":"2024-05-08T08:27:53.544667+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'805","last_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_deep_scrub":"158'805","last_deep_scrub_stamp":"2024-05-08T08:27:46.131458+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.131458+0000","objects_scrubbed":8,"log_size":805,"log_dups_size":0,"ondisk_log_size":805,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:20:12.279284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00058966699999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":7,"num_read":6674,"num_read_kb":37171,"num_write":869,"num_write_kb":25736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":124,"num_bytes_recovered":659757,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":162,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547025+0000","last_change":"2024-05-08T08:27:47.420768+0000","last_active":"2024-05-08T08:27:53.547025+0000","last_peered":"2024-05-08T08:27:53.547025+0000","last_clean":"2024-05-08T08:27:53.547025+0000","last_became_active":"2024-05-08T08:27:29.992683+0000","last_became_peered":"2024-05-08T08:27:29.992683+0000","last_unstale":"2024-05-08T08:27:53.547025+0000","last_undegraded":"2024-05-08T08:27:53.547025+0000","last_fullsized":"2024-05-08T08:27:53.547025+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:47.420622+0000","last_clean_scrub_stamp":"2024-05-08T08:27:47.420622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:10:16.709381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"163'338","reported_seq":2013,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:30:11.746797+0000","last_change":"2024-05-08T08:30:11.746834+0000","last_active":"2024-05-08T08:30:11.746797+0000","last_peered":"2024-05-08T08:30:11.746797+0000","last_clean":"2024-05-08T08:30:11.746797+0000","last_became_active":"2024-05-08T08:27:53.548879+0000","last_became_peered":"2024-05-08T08:27:53.548879+0000","last_unstale":"2024-05-08T08:30:11.746797+0000","last_undegraded":"2024-05-08T08:30:11.746797+0000","last_fullsized":"2024-05-08T08:30:11.746797+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'338","last_scrub_stamp":"2024-05-08T08:30:11.746700+0000","last_deep_scrub":"163'338","last_deep_scrub_stamp":"2024-05-08T08:30:11.746700+0000","last_clean_scrub_stamp":"2024-05-08T08:30:11.746700+0000","objects_scrubbed":14,"log_size":338,"log_dups_size":0,"ondisk_log_size":338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:07:52.013409+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.0014390360000000001,"stat_sum":{"num_bytes":0,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":14,"num_read":1239,"num_read_kb":48449,"num_write":293,"num_write_kb":21427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":317,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546796+0000","last_change":"2024-05-08T08:27:49.302530+0000","last_active":"2024-05-08T08:27:53.546796+0000","last_peered":"2024-05-08T08:27:53.546796+0000","last_clean":"2024-05-08T08:27:53.546796+0000","last_became_active":"2024-05-08T08:24:56.680213+0000","last_became_peered":"2024-05-08T08:24:56.680213+0000","last_unstale":"2024-05-08T08:27:53.546796+0000","last_undegraded":"2024-05-08T08:27:53.546796+0000","last_fullsized":"2024-05-08T08:27:53.546796+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T08:27:49.302415+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.302415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:57:45.433494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"163'465","reported_seq":5247,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:30:12.758201+0000","last_change":"2024-05-08T08:30:12.758241+0000","last_active":"2024-05-08T08:30:12.758201+0000","last_peered":"2024-05-08T08:30:12.758201+0000","last_clean":"2024-05-08T08:30:12.758201+0000","last_became_active":"2024-05-08T08:27:53.548297+0000","last_became_peered":"2024-05-08T08:27:53.548297+0000","last_unstale":"2024-05-08T08:30:12.758201+0000","last_undegraded":"2024-05-08T08:30:12.758201+0000","last_fullsized":"2024-05-08T08:30:12.758201+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'465","last_scrub_stamp":"2024-05-08T08:30:12.758112+0000","last_deep_scrub":"163'465","last_deep_scrub_stamp":"2024-05-08T08:30:12.758112+0000","last_clean_scrub_stamp":"2024-05-08T08:30:12.758112+0000","objects_scrubbed":9,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:34:54.425548+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.002976154,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":6119,"num_read_kb":26697,"num_write":448,"num_write_kb":20644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":293,"num_bytes_recovered":3761146,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"158'362","reported_seq":1620,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.634696+0000","last_change":"2024-05-08T08:27:46.589568+0000","last_active":"2024-05-08T08:27:53.634696+0000","last_peered":"2024-05-08T08:27:53.634696+0000","last_clean":"2024-05-08T08:27:53.634696+0000","last_became_active":"2024-05-08T08:27:23.143860+0000","last_became_peered":"2024-05-08T08:27:23.143860+0000","last_unstale":"2024-05-08T08:27:53.634696+0000","last_undegraded":"2024-05-08T08:27:53.634696+0000","last_fullsized":"2024-05-08T08:27:53.634696+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'362","last_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_deep_scrub":"158'362","last_deep_scrub_stamp":"2024-05-08T08:27:46.589438+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.589438+0000","objects_scrubbed":9,"log_size":362,"log_dups_size":0,"ondisk_log_size":362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:09:36.175812+0000","scrub_duration":6,"objects_trimmed":14,"snaptrim_duration":0.067690549000000003,"stat_sum":{"num_bytes":0,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":9,"num_read":2540,"num_read_kb":7582,"num_write":325,"num_write_kb":4382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"160'678","reported_seq":2016,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.545099+0000","last_change":"2024-05-08T08:27:51.271464+0000","last_active":"2024-05-08T08:27:53.545099+0000","last_peered":"2024-05-08T08:27:53.545099+0000","last_clean":"2024-05-08T08:27:53.545099+0000","last_became_active":"2024-05-08T08:27:28.181624+0000","last_became_peered":"2024-05-08T08:27:28.181624+0000","last_unstale":"2024-05-08T08:27:53.545099+0000","last_undegraded":"2024-05-08T08:27:53.545099+0000","last_fullsized":"2024-05-08T08:27:53.545099+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'678","last_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_deep_scrub":"160'678","last_deep_scrub_stamp":"2024-05-08T08:27:51.271332+0000","last_clean_scrub_stamp":"2024-05-08T08:27:51.271332+0000","objects_scrubbed":10,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:59:43.744439+0000","scrub_duration":9,"objects_trimmed":12,"snaptrim_duration":0.027202541,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":9,"num_read":2253,"num_read_kb":18963,"num_write":828,"num_write_kb":10109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":122,"num_bytes_recovered":7616040,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.b","version":"158'378","reported_seq":2604,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.544495+0000","last_change":"2024-05-08T08:27:49.298294+0000","last_active":"2024-05-08T08:27:53.544495+0000","last_peered":"2024-05-08T08:27:53.544495+0000","last_clean":"2024-05-08T08:27:53.544495+0000","last_became_active":"2024-05-08T08:27:28.177989+0000","last_became_peered":"2024-05-08T08:27:28.177989+0000","last_unstale":"2024-05-08T08:27:53.544495+0000","last_undegraded":"2024-05-08T08:27:53.544495+0000","last_fullsized":"2024-05-08T08:27:53.544495+0000","mapping_epoch":148,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":149,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'378","last_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_deep_scrub":"158'378","last_deep_scrub_stamp":"2024-05-08T08:27:49.298146+0000","last_clean_scrub_stamp":"2024-05-08T08:27:49.298146+0000","objects_scrubbed":10,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:19:37.338005+0000","scrub_duration":8,"objects_trimmed":6,"snaptrim_duration":0.019603117,"stat_sum":{"num_bytes":0,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":10,"num_read":1815,"num_read_kb":17245,"num_write":332,"num_write_kb":11350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.c","version":"159'381","reported_seq":4596,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547282+0000","last_change":"2024-05-08T08:27:50.358687+0000","last_active":"2024-05-08T08:27:53.547282+0000","last_peered":"2024-05-08T08:27:53.547282+0000","last_clean":"2024-05-08T08:27:53.547282+0000","last_became_active":"2024-05-08T08:27:29.995606+0000","last_became_peered":"2024-05-08T08:27:29.995606+0000","last_unstale":"2024-05-08T08:27:53.547282+0000","last_undegraded":"2024-05-08T08:27:53.547282+0000","last_fullsized":"2024-05-08T08:27:53.547282+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'381","last_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_deep_scrub":"159'381","last_deep_scrub_stamp":"2024-05-08T08:27:50.358500+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.358500+0000","objects_scrubbed":8,"log_size":381,"log_dups_size":0,"ondisk_log_size":381,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:29:30.894854+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.0013804939999999999,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":4487,"num_read_kb":41623,"num_write":413,"num_write_kb":24642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.d","version":"159'378","reported_seq":1908,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547267+0000","last_change":"2024-05-08T08:27:50.270819+0000","last_active":"2024-05-08T08:27:53.547267+0000","last_peered":"2024-05-08T08:27:53.547267+0000","last_clean":"2024-05-08T08:27:53.547267+0000","last_became_active":"2024-05-08T08:25:04.605464+0000","last_became_peered":"2024-05-08T08:25:04.605464+0000","last_unstale":"2024-05-08T08:27:53.547267+0000","last_undegraded":"2024-05-08T08:27:53.547267+0000","last_fullsized":"2024-05-08T08:27:53.547267+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'378","last_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_deep_scrub":"159'378","last_deep_scrub_stamp":"2024-05-08T08:27:50.270696+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.270696+0000","objects_scrubbed":5,"log_size":378,"log_dups_size":0,"ondisk_log_size":378,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:18:50.064149+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00074273399999999999,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":5,"num_read":1518,"num_read_kb":13493,"num_write":337,"num_write_kb":6486,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.e","version":"163'671","reported_seq":5078,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:54.285076+0000","last_change":"2024-05-08T08:27:54.285076+0000","last_active":"2024-05-08T08:27:54.285076+0000","last_peered":"2024-05-08T08:27:54.285076+0000","last_clean":"2024-05-08T08:27:54.285076+0000","last_became_active":"2024-05-08T08:27:24.155355+0000","last_became_peered":"2024-05-08T08:27:24.155355+0000","last_unstale":"2024-05-08T08:27:54.285076+0000","last_undegraded":"2024-05-08T08:27:54.285076+0000","last_fullsized":"2024-05-08T08:27:54.285076+0000","mapping_epoch":144,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":145,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'671","last_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_deep_scrub":"163'671","last_deep_scrub_stamp":"2024-05-08T08:27:54.284897+0000","last_clean_scrub_stamp":"2024-05-08T08:27:54.284897+0000","objects_scrubbed":11,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:32:38.548352+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.0013795280000000001,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":6437,"num_read_kb":47689,"num_write":652,"num_write_kb":27386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":157,"num_bytes_recovered":2139747,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"161'463","reported_seq":1953,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547462+0000","last_change":"2024-05-08T08:27:52.288279+0000","last_active":"2024-05-08T08:27:53.547462+0000","last_peered":"2024-05-08T08:27:53.547462+0000","last_clean":"2024-05-08T08:27:53.547462+0000","last_became_active":"2024-05-08T08:27:23.150603+0000","last_became_peered":"2024-05-08T08:27:23.150603+0000","last_unstale":"2024-05-08T08:27:53.547462+0000","last_undegraded":"2024-05-08T08:27:53.547462+0000","last_fullsized":"2024-05-08T08:27:53.547462+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"161'463","last_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_deep_scrub":"161'463","last_deep_scrub_stamp":"2024-05-08T08:27:52.288116+0000","last_clean_scrub_stamp":"2024-05-08T08:27:52.288116+0000","objects_scrubbed":6,"log_size":463,"log_dups_size":0,"ondisk_log_size":463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:01:29.656738+0000","scrub_duration":5,"objects_trimmed":8,"snaptrim_duration":0.022899085,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":6,"num_read":1854,"num_read_kb":7519,"num_write":554,"num_write_kb":8601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.10","version":"159'700","reported_seq":3091,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546382+0000","last_change":"2024-05-08T08:27:50.089182+0000","last_active":"2024-05-08T08:27:53.546382+0000","last_peered":"2024-05-08T08:27:53.546382+0000","last_clean":"2024-05-08T08:27:53.546382+0000","last_became_active":"2024-05-08T08:27:08.081145+0000","last_became_peered":"2024-05-08T08:27:08.081145+0000","last_unstale":"2024-05-08T08:27:53.546382+0000","last_undegraded":"2024-05-08T08:27:53.546382+0000","last_fullsized":"2024-05-08T08:27:53.546382+0000","mapping_epoch":131,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":132,"parent":"0.0","parent_split_bits":0,"last_scrub":"159'700","last_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_deep_scrub":"159'700","last_deep_scrub_stamp":"2024-05-08T08:27:50.089001+0000","last_clean_scrub_stamp":"2024-05-08T08:27:50.089001+0000","objects_scrubbed":2,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:30:28.409907+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00078459500000000002,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":2,"num_read":3824,"num_read_kb":12531,"num_write":982,"num_write_kb":9123,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":547596,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.11","version":"162'472","reported_seq":1904,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.547075+0000","last_change":"2024-05-08T08:27:53.303671+0000","last_active":"2024-05-08T08:27:53.547075+0000","last_peered":"2024-05-08T08:27:53.547075+0000","last_clean":"2024-05-08T08:27:53.547075+0000","last_became_active":"2024-05-08T08:27:23.151650+0000","last_became_peered":"2024-05-08T08:27:23.151650+0000","last_unstale":"2024-05-08T08:27:53.547075+0000","last_undegraded":"2024-05-08T08:27:53.547075+0000","last_fullsized":"2024-05-08T08:27:53.547075+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'472","last_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_deep_scrub":"162'472","last_deep_scrub_stamp":"2024-05-08T08:27:53.303507+0000","last_clean_scrub_stamp":"2024-05-08T08:27:53.303507+0000","objects_scrubbed":8,"log_size":472,"log_dups_size":0,"ondisk_log_size":472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:58:10.890252+0000","scrub_duration":12,"objects_trimmed":12,"snaptrim_duration":0.026789770000000001,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":8,"num_read":1722,"num_read_kb":20588,"num_write":524,"num_write_kb":8871,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"158'386","reported_seq":3493,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:53.546746+0000","last_change":"2024-05-08T08:27:46.047343+0000","last_active":"2024-05-08T08:27:53.546746+0000","last_peered":"2024-05-08T08:27:53.546746+0000","last_clean":"2024-05-08T08:27:53.546746+0000","last_became_active":"2024-05-08T08:27:02.041573+0000","last_became_peered":"2024-05-08T08:27:02.041573+0000","last_unstale":"2024-05-08T08:27:53.546746+0000","last_undegraded":"2024-05-08T08:27:53.546746+0000","last_fullsized":"2024-05-08T08:27:53.546746+0000","mapping_epoch":125,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":126,"parent":"0.0","parent_split_bits":0,"last_scrub":"158'386","last_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_deep_scrub":"158'386","last_deep_scrub_stamp":"2024-05-08T08:27:46.047238+0000","last_clean_scrub_stamp":"2024-05-08T08:27:46.047238+0000","objects_scrubbed":11,"log_size":386,"log_dups_size":0,"ondisk_log_size":386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:22.707888+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00085707299999999997,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":11,"num_read":2793,"num_read_kb":31186,"num_write":409,"num_write_kb":14914,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":192,"num_bytes_recovered":17552525,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"163'713","reported_seq":4735,"reported_epoch":163,"state":"active+clean","last_fresh":"2024-05-08T08:27:55.277400+0000","last_change":"2024-05-08T08:27:55.277400+0000","last_active":"2024-05-08T08:27:55.277400+0000","last_peered":"2024-05-08T08:27:55.277400+0000","last_clean":"2024-05-08T08:27:55.277400+0000","last_became_active":"2024-05-08T08:27:53.548858+0000","last_became_peered":"2024-05-08T08:27:53.548858+0000","last_unstale":"2024-05-08T08:27:55.277400+0000","last_undegraded":"2024-05-08T08:27:55.277400+0000","last_fullsized":"2024-05-08T08:27:55.277400+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"163'713","last_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_deep_scrub":"163'713","last_deep_scrub_stamp":"2024-05-08T08:27:55.277274+0000","last_clean_scrub_stamp":"2024-05-08T08:27:55.277274+0000","objects_scrubbed":7,"log_size":713,"log_dups_size":0,"ondisk_log_size":713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:33.322747+0000","scrub_duration":5,"objects_trimmed":22,"snaptrim_duration":0.046055342999999999,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":7,"num_read":5311,"num_read_kb":19404,"num_write":990,"num_write_kb":14296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":132,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":277,"num_object_clones":0,"num_object_copies":554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":277,"num_whiteouts":273,"num_read":158279,"num_read_kb":861606,"num_write":21846,"num_write_kb":472108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3359,"num_bytes_recovered":71491886,"num_keys_recovered":92,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26071,"internal_metadata":0},"log_size":18618,"ondisk_log_size":18618,"up":64,"acting":64,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":9886,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574919,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55356,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316484,"statfs":{"total":96636764160,"available":96580079616,"internally_reserved":0,"allocated":2019328,"data_stored":1545508,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":54655434},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574919,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28944,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94342896,"statfs":{"total":96636764160,"available":96607125504,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574919,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55300,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316540,"statfs":{"total":96636764160,"available":96580136960,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":143,"seq":614180323367,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35468,"kb_used_data":1920,"kb_used_omap":32,"kb_used_meta":33503,"kb_avail":94336372,"statfs":{"total":96636764160,"available":96600444928,"internally_reserved":0,"allocated":1966080,"data_stored":955159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33708,"internal_metadata":34307156},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55300,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316540,"statfs":{"total":96636764160,"available":96580136960,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37256,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334584,"statfs":{"total":96636764160,"available":96598614016,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":54,"seq":231928234043,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43136,"kb_used_data":1976,"kb_used_omap":13,"kb_used_meta":41138,"kb_avail":94328704,"statfs":{"total":96636764160,"available":96592592896,"internally_reserved":0,"allocated":2023424,"data_stored":1545527,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":13550,"internal_metadata":42126098},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574919,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":77508,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":75582,"kb_avail":94294332,"statfs":{"total":96636764160,"available":96557395968,"internally_reserved":0,"allocated":1961984,"data_stored":955140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":77396421},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":4943,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9107,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":16964,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T08:30:31.530 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T08:30:31.844 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T08:30:31.844 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T08:30:31.844 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T08:30:31.844 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:31.882 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T08:30:31.882 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T08:30:31.882 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:31.955 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T08:30:31.955 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T08:30:31.955 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.023 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T08:30:32.023 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T08:30:32.023 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.095 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T08:30:32.095 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T08:30:32.095 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.127 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T08:30:32.127 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T08:30:32.127 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.202 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T08:30:32.203 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T08:30:32.203 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.285 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T08:30:32.285 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T08:30:32.285 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.329 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T08:30:32.329 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T08:30:32.329 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T08:30:32.329 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.354 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T08:30:32.355 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T08:30:32.355 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.383 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T08:30:32.384 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T08:30:32.384 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T08:30:32.384 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.406 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T08:30:32.406 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T08:30:32.406 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.457 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T08:30:32.458 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T08:30:32.458 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T08:30:32.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-05-08T08:30:32.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-05-08T08:30:32.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2024-05-08T08:30:32.470 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2024-05-08T08:30:32.470 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T08:30:32.471 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T08:30:32.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-05-08T08:30:32.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:30:32.532 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T08:30:32.733 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:30:32.733 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T08:30:32.848 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:30:32.848 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T08:30:32.911 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:30:32.911 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T08:30:32.962 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:30:32.962 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T08:30:33.072 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:30:33.073 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T08:30:33.135 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:30:33.136 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T08:30:33.192 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:30:33.192 DEBUG:teuthology.orchestra.run.smithi149:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T08:30:33.251 INFO:tasks.ceph:Archiving mon data... 2024-05-08T08:30:33.253 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/data/mon.a.tgz 2024-05-08T08:30:33.253 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-05-08T08:30:33.269 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.uoUIZi6pnT 2024-05-08T08:30:33.270 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uoUIZi6pnT 2024-05-08T08:30:33.435 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.uoUIZi6pnT 2024-05-08T08:30:33.521 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.uoUIZi6pnT is 420KB 2024-05-08T08:30:33.617 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.uoUIZi6pnT 2024-05-08T08:30:33.633 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/data/mon.c.tgz 2024-05-08T08:30:33.633 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-05-08T08:30:33.688 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.0mCpxiEGUr 2024-05-08T08:30:33.688 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.0mCpxiEGUr 2024-05-08T08:30:33.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.0mCpxiEGUr 2024-05-08T08:30:33.958 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.0mCpxiEGUr is 565KB 2024-05-08T08:30:34.031 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.0mCpxiEGUr 2024-05-08T08:30:34.047 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/data/mon.b.tgz 2024-05-08T08:30:34.047 DEBUG:teuthology.orchestra.run.smithi149:> mktemp 2024-05-08T08:30:34.066 INFO:teuthology.orchestra.run.smithi149.stdout:/tmp/tmp.LFQcJRJ99E 2024-05-08T08:30:34.066 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.LFQcJRJ99E 2024-05-08T08:30:34.298 DEBUG:teuthology.orchestra.run.smithi149:> sudo chmod 0666 /tmp/tmp.LFQcJRJ99E 2024-05-08T08:30:34.383 DEBUG:teuthology.orchestra.remote:smithi149:/tmp/tmp.LFQcJRJ99E is 566KB 2024-05-08T08:30:34.482 DEBUG:teuthology.orchestra.run.smithi149:> rm -fr /tmp/tmp.LFQcJRJ99E 2024-05-08T08:30:34.497 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T08:30:34.497 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T08:30:34.502 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T08:30:34.583 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T08:30:34.617 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T08:30:34.617 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T08:30:34.619 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/remote/smithi031/crash 2024-05-08T08:30:34.619 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T08:30:34.651 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/remote/smithi149/crash 2024-05-08T08:30:34.652 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T08:30:34.681 INFO:tasks.ceph:Compressing logs... 2024-05-08T08:30:34.681 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T08:30:34.696 DEBUG:teuthology.orchestra.run.smithi149:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T08:30:34.721 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56697.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32331.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.32331.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32331.log.gz 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42754.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/objectstore_tool.42754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32120.log 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr: 85.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T08:30:34.778 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.32120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32120.log.gz 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr: 89.2%gzip -- replaced with /var/log/ceph/ceph.audit.log.gz -5 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32251.log 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32211.log 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.32251.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32251.log.gz 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32080.log 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.32211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32211.log.gz 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-08T08:30:34.779 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.32080.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32080.log.gz 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.32171.log 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32291.log 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.32171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32171.log.gzgzip -5 --verbose -- /var/log/ceph/objectstore_tool.43609.log 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.32291.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32291.log.gz 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.45343.log 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/objectstore_tool.43609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32040.log 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/objectstore_tool.45343.log: 94.9% -- replaced with /var/log/ceph/objectstore_tool.42754.log.gz 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.44468.log 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-client.admin.32040.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32040.log.gz 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/objectstore_tool.44468.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.43609.log.gz 2024-05-08T08:30:34.780 INFO:teuthology.orchestra.run.smithi149.stderr:/var/log/ceph/ceph-mon.b.log: 94.7% -- replaced with /var/log/ceph/objectstore_tool.45343.log.gz 2024-05-08T08:30:34.781 INFO:teuthology.orchestra.run.smithi149.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.44468.log.gz 2024-05-08T08:30:34.818 INFO:teuthology.orchestra.run.smithi149.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T08:30:34.895 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52177.log 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54476.log 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56697.log.gz 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57255.log 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52177.log.gz 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57941.log 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54476.log.gz 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60196.log 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57255.log.gz 2024-05-08T08:30:34.896 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51900.log 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57941.log.gz 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60196.log.gz 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60112.log 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51900.log.gz 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36655.log 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58612.log 2024-05-08T08:30:34.897 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60112.log.gz 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50522.log 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36655.log.gz 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47769.log 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58612.log.gz 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50522.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41682.log 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50522.log.gz 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59523.log 2024-05-08T08:30:34.898 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47769.log.gz 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/objectstore_tool.41682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44070.log 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59565.log 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59523.log.gz 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44070.log.gz 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55186.log 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57381.log 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59565.log.gz 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55186.log.gz 2024-05-08T08:30:34.899 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48368.log 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57087.log 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57381.log.gz 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48368.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36663.log 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48368.log.gz 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46050.log 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57087.log.gz 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36663.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36663.log.gz 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37621.log 2024-05-08T08:30:34.900 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58277.log 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46050.log.gz 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58493.log 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37621.log.gz 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50983.log 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58277.log.gz 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51839.log 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58493.log.gz 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50983.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.50983.log.gz --verbose 2024-05-08T08:30:34.901 INFO:teuthology.orchestra.run.smithi031.stderr: -- /var/log/ceph/ceph.log 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr: 95.0% -- replaced with /var/log/ceph/objectstore_tool.41682.log.gz 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54847.log 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51839.log.gz 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59817.log 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43512.log 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54847.log.gz 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59061.log 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59817.log.gz 2024-05-08T08:30:34.902 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57465.log 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43512.log.gz 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49823.log 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59061.log.gz 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52220.log 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57465.log.gz 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr: 92.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49823.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49823.log.gz 2024-05-08T08:30:34.903 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49384.log 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52220.log.gz 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60154.log 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46614.log 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49384.log.gz 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60154.log.gz 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59103.log 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37946.log 2024-05-08T08:30:34.904 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46614.log.gz 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57297.log 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr: 25.6% -- replaced with /var/log/ceph/ceph-client.admin.59103.log.gz 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.0.37946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55308.log 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47727.log 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57297.log.gz 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55308.log: 95.2% -- replaced with /var/log/ceph/ceph-client.0.37946.log.gz 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.55308.log.gz 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph.audit.log 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47366.log 2024-05-08T08:30:34.905 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47727.log.gz 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60278.log 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51782.log 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47366.log.gz 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60278.log.gz 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39229.log 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54078.log 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51782.log.gz 2024-05-08T08:30:34.906 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39229.log.gz 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57633.log 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49741.log 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54078.log.gz 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57633.log.gz 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53557.log 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39439.log 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49741.log.gz 2024-05-08T08:30:34.907 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56956.log 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53557.log.gz 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57591.log 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39439.log.gz 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57937.log 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56956.log.gz 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58613.log 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57591.log.gz 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58269.log 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57937.log.gz 2024-05-08T08:30:34.908 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59271.log 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58613.log.gz 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58275.log 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58269.log.gz 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53123.log 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59271.log.gz 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43457.log 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58275.log.gz 2024-05-08T08:30:34.909 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38031.log 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53123.log.gz 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49045.log 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43457.log.gz 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37344.log 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38031.log.gz 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50882.log 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49045.log.gz 2024-05-08T08:30:34.910 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37344.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37344.log.gz 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36658.log 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50882.log.gz 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37747.log 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58276.log 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36658.log.gz 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50922.log 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37747.log.gz 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58276.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58276.log.gz 2024-05-08T08:30:34.911 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47079.log 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38749.log 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50922.log.gz 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47079.log.gz 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59901.log 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39707.log 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38749.log.gz 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59901.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59901.log.gz 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50303.log 2024-05-08T08:30:34.912 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57935.log 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39707.log.gz 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50303.log.gz 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55957.log 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39110.log 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57935.log.gz 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55957.log.gz 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48546.log 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36650.log 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39110.log.gz 2024-05-08T08:30:34.913 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48546.log.gz 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43789.log 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36650.log.gz 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43789.log.gz 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36353.log 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49522.log 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36353.log.gz 2024-05-08T08:30:34.914 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37893.log 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40932.log 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49522.log.gz 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37893.log.gz 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37320.log 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.40932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46331.log 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40932.log.gz 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37320.log.gz 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50982.log 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51560.log 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46331.log.gz 2024-05-08T08:30:34.915 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50982.log.gz 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57339.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51560.log.gz 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47548.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57339.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57339.log.gz 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32882.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47548.log.gz 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44524.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.32882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32882.log.gz 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59145.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57940.log 2024-05-08T08:30:34.916 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44524.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44524.log.gz 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48150.log 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59145.log.gz 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54415.log 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57940.log.gz 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36255.log 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48150.log.gz 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39905.log 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54415.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54415.log.gz 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50822.log 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36255.log.gz 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59649.log 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39905.log.gz 2024-05-08T08:30:34.917 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60238.log 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50822.log.gz 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59649.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59649.log.gz 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60238.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.60238.log.gz /var/log/ceph/ceph-client.admin.51401.log 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47303.log 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.51401.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51401.log.gz 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.37407.log 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47303.log.gz 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47201.log 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37407.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37407.log.gz 2024-05-08T08:30:34.918 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50582.log 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47201.log.gz 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54807.log 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50582.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50582.log.gz 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.53790.log 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54807.log.gz 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36660.log 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37323.log 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53790.log.gz 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36660.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36660.log.gz 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47665.log 2024-05-08T08:30:34.919 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37323.log.gz 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48008.log 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44860.log 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47665.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47665.log.gz 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55563.log 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48008.log.gz 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60027.log 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44860.log.gz 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.55563.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.59859.log 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55563.log.gz 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.60027.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60027.log.gz 2024-05-08T08:30:34.920 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36937.log 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59859.log.gz 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58893.log 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.36937.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36937.log.gz 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58893.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59607.log 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58893.log.gz 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39351.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50702.log 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59019.log 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59607.log.gz 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr:gzip/var/log/ceph/ceph-client.admin.50702.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57129.log 2024-05-08T08:30:34.921 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50702.log.gz 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.59019.log: /var/log/ceph/ceph-client.admin.57045.log 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59019.log.gz/var/log/ceph/ceph-client.admin.57129.log: 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57129.log.gz 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr: -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57045.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57045.log.gz 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42519.log 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56998.log/var/log/ceph/ceph-mon.a.log: 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/objectstore_tool.42519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46092.log 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56998.log.gz 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36662.log 2024-05-08T08:30:34.922 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46092.log.gz 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38632.log 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36662.log.gz 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50762.log 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38632.log.gz 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42461.log 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50762.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50762.log.gz 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56294.log 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.42461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42461.log.gz 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58935.log 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56294.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56294.log.gz 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44192.log 2024-05-08T08:30:34.923 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58935.log.gz 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55523.log 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44192.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44192.log.gz 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.42519.log.gz 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37094.log 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37094.log: /var/log/ceph/ceph-client.admin.55523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55523.log.gz 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37094.log.gz 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38791.log 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49865.log 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.924 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56079.log 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38791.log: /var/log/ceph/ceph-client.admin.49865.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38791.log.gz 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49865.log.gz 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43850.log 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56079.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56079.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59313.log 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57939.log 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43850.log: /var/log/ceph/ceph-client.admin.59313.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43850.log.gz 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59313.log.gz 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37243.log 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57939.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39171.log 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57939.log.gz 2024-05-08T08:30:34.925 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55605.log 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37243.log: /var/log/ceph/ceph-client.admin.39171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37243.log.gz 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39171.log.gz 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59355.log 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55605.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48267.log 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36661.log 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59355.log.gz 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48267.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36918.log 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48267.log.gz 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46887.log 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36661.log: /var/log/ceph/ceph-client.admin.36918.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36661.log.gz 2024-05-08T08:30:34.926 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36918.log.gz 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58137.log 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46887.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36936.log 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57507.log 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58137.log.gz 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36936.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60070.log 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36936.log.gz 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44429.log 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57507.log: /var/log/ceph/ceph-client.admin.60070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60070.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57507.log.gz 2024-05-08T08:30:34.927 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47122.log 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44429.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37831.log 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52837.log 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47122.log.gz 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37831.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59187.log 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37831.log.gz 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59691.log 2024-05-08T08:30:34.928 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52837.log: /var/log/ceph/ceph-client.admin.59187.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52837.log.gz 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59187.log.gz 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46694.log 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59691.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47886.log 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51182.log 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46694.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46694.log.gz 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47886.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26658.log 2024-05-08T08:30:34.929 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47886.log.gz 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52459.log 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51182.log.gz 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.tmp-client.admin.26658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36939.log 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26658.log.gz 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56419.log 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52459.log.gz 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36939.log.gz 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52596.log 2024-05-08T08:30:34.930 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56419.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52939.log 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53260.log 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52596.log.gz 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52939.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.50365.log 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.52939.log.gz 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57171.log 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53260.log.gz 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38407.log 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50365.log.gz 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36987.log 2024-05-08T08:30:34.931 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57171.log.gz 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38407.log.gz 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38350.log 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36987.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.36987.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.58809.log 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55071.log 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38350.log.gz 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58809.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39610.log 2024-05-08T08:30:34.932 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58809.log.gz 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57931.log 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55071.log.gz 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.39610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38032.log 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39610.log.gz 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46515.log 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.38032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57931.log.gz 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38032.log.gz 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47141.log 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46515.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57936.log 2024-05-08T08:30:34.933 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46515.log.gz 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56914.log 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47141.log.gz 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43846.log 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57936.log.gz 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57549.log 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56914.log.gz 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43846.log.gz 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57759.log 2024-05-08T08:30:34.934 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57549.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51458.log 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57549.log.gz 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59775.log 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57759.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57759.log.gz 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51458.log: gzip 0.0% -5 --verbose -- /var/log/ceph/ceph-client.admin.48587.log 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.51458.log.gz 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37789.log 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59775.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59775.log.gz 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50186.log 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48587.log.gz 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59397.log 2024-05-08T08:30:34.935 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37789.log.gz 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50186.log.gz 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52755.log 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59397.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58851.log 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52755.log.gz 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49906.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46330.log 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48637.log 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58851.log.gz 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46695.log 2024-05-08T08:30:34.936 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46330.log.gz 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59943.log 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48637.log.gz 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58274.log 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46695.log.gz 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46818.log 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59943.log.gz 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58274.log.gz 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59229.log 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46818.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53340.log 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.937 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59733.log 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59229.log.gz 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57213.log 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53340.log.gz 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58610.log 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59733.log.gz 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57213.log.gz 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44232.log 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58610.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58610.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47256.log 2024-05-08T08:30:34.938 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36988.log 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.47256.log: /var/log/ceph/ceph-client.admin.44232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44232.log.gz 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47256.log.gz 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44606.log 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36988.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58614.log 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36988.log.gz 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56274.log 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44606.log: /var/log/ceph/ceph-client.admin.58614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44606.log.gz 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58614.log.gz 2024-05-08T08:30:34.939 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59481.log 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56274.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36397.log 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37705.log 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36397.log: /var/log/ceph/ceph-client.admin.59481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59481.log.gz 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36397.log.gz 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58977.log 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37705.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37705.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.37663.log 2024-05-08T08:30:34.940 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45829.log 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58977.log: /var/log/ceph/ceph-client.admin.37663.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58977.log.gz 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37663.log.gz 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37579.log 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.45829.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46152.log 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45829.log.gz 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53519.log 2024-05-08T08:30:34.941 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37579.log.gz 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37321.log 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46152.log.gz 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.53519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52260.log 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37321.log: 28.8% -- replaced with /var/log/ceph/ceph-client.admin.53519.log.gz 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37321.log.gz 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58268.log 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52260.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.52260.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.43714.log 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32477.log 2024-05-08T08:30:34.942 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58268.log.gz 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59439.log 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43714.log.gz 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.32477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48890.log 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32477.log.gz 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59439.log.gz 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48984.log 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.48890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48890.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49303.log 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51279.log 2024-05-08T08:30:34.943 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49303.log: /var/log/ceph/ceph-client.admin.48984.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48984.log.gz 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49303.log.gz 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36311.log 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.51279.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51279.log.gzgzip -5 --verbose -- /var/log/ceph/objectstore_tool.40703.log 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36945.log 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36311.log: /var/log/ceph/objectstore_tool.40703.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36311.log.gz 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57675.log 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36945.log.gz 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43988.log 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57675.log.gz 2024-05-08T08:30:34.944 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57717.log 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.43988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43988.log.gz 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59985.log 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57717.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57717.log.gz 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55882.log 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.59985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59985.log.gz 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56461.log 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.55882.log: 95.1% -- replaced with /var/log/ceph/objectstore_tool.40703.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55882.log.gz 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52658.log 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56461.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36654.log 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.945 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32779.log 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52658.log.gz 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36654.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56377.log 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.36654.log.gz 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56778.log 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.32779.log: /var/log/ceph/ceph-client.admin.56377.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32779.log.gz 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56377.log.gz 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54752.log 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.56778.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52999.log 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56778.log.gz 2024-05-08T08:30:34.946 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58473.log 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54752.log.gz 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.52999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58271.log 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52999.log.gz 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36481.log 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58473.log.gz 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58271.log.gz 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58615.log 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36481.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37322.log 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:34.947 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.39923.log 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58615.log: /var/log/ceph/ceph-client.admin.37322.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58615.log.gz 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37322.log.gz 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44907.log 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/objectstore_tool.39923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50642.log 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.44907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44907.log.gz 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46556.log 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.50642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50642.log.gz 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49964.log 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.46556.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46556.log.gz 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57423.log 2024-05-08T08:30:34.948 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.49964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49964.log.gz 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58611.log 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57423.log.gz 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57938.log 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.58611.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58611.log.gz 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54260.log 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.57938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57938.log.gz 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37405.log 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.54260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54260.log.gz 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36439.log 2024-05-08T08:30:34.949 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.37405.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37405.log.gz 2024-05-08T08:30:34.950 INFO:teuthology.orchestra.run.smithi031.stderr: 94.9% -- replaced with /var/log/ceph/objectstore_tool.39923.log.gz 2024-05-08T08:30:34.950 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.36439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36439.log.gz 2024-05-08T08:30:34.950 INFO:teuthology.orchestra.run.smithi031.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T08:30:34.986 INFO:teuthology.orchestra.run.smithi149.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T08:30:35.191 INFO:teuthology.orchestra.run.smithi031.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T08:30:35.247 INFO:teuthology.orchestra.run.smithi031.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T08:30:35.572 INFO:teuthology.orchestra.run.smithi149.stderr: 94.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-08T08:30:36.094 INFO:teuthology.orchestra.run.smithi031.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T08:30:36.768 INFO:teuthology.orchestra.run.smithi149.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T08:30:37.163 INFO:teuthology.orchestra.run.smithi149.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-08T08:30:37.471 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-05-08T08:30:37.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-05-08T08:30:37.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2024-05-08T08:30:37.472 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2024-05-08T08:30:37.785 INFO:teuthology.orchestra.run.smithi031.stderr: 94.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T08:30:37.944 INFO:teuthology.orchestra.run.smithi149.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T08:30:37.946 INFO:teuthology.orchestra.run.smithi149.stderr: 2024-05-08T08:30:37.946 INFO:teuthology.orchestra.run.smithi149.stderr:real 0m3.205s 2024-05-08T08:30:37.946 INFO:teuthology.orchestra.run.smithi149.stderr:user 0m8.422s 2024-05-08T08:30:37.946 INFO:teuthology.orchestra.run.smithi149.stderr:sys 0m0.349s 2024-05-08T08:30:38.708 INFO:teuthology.orchestra.run.smithi031.stderr: 95.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T08:30:38.711 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-05-08T08:30:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:real 0m4.000s 2024-05-08T08:30:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:user 0m9.236s 2024-05-08T08:30:38.711 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m0.498s 2024-05-08T08:30:38.711 INFO:tasks.ceph:Archiving logs... 2024-05-08T08:30:38.711 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/remote/smithi031/log 2024-05-08T08:30:38.712 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T08:30:40.092 DEBUG:teuthology.misc:Transferring archived files from smithi149:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/remote/smithi149/log 2024-05-08T08:30:40.093 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T08:30:41.187 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T08:30:41.196 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T08:30:41.196 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T08:30:41.199 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T08:30:41.280 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T08:30:41.290 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T08:30:41.290 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:30:41.292 DEBUG:teuthology.orchestra.run.smithi149:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T08:30:41.307 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:30:41.309 INFO:teuthology.orchestra.run.smithi149.stderr:bash: line 1: ntpq: command not found 2024-05-08T08:30:41.312 INFO:teuthology.orchestra.run.smithi031.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:30:41.312 INFO:teuthology.orchestra.run.smithi031.stdout:=============================================================================== 2024-05-08T08:30:41.312 INFO:teuthology.orchestra.run.smithi031.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 -1493us[-1473us] +/- 67ms 2024-05-08T08:30:41.312 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +342us[ +342us] +/- 73ms 2024-05-08T08:30:41.312 INFO:teuthology.orchestra.run.smithi031.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +1483us[+1483us] +/- 81ms 2024-05-08T08:30:41.312 INFO:teuthology.orchestra.run.smithi031.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:30:41.314 INFO:teuthology.orchestra.run.smithi149.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T08:30:41.314 INFO:teuthology.orchestra.run.smithi149.stdout:=============================================================================== 2024-05-08T08:30:41.314 INFO:teuthology.orchestra.run.smithi149.stdout:^* hv01.front.sepia.ceph.com 3 6 377 14 -1489us[-1448us] +/- 67ms 2024-05-08T08:30:41.314 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +351us[ +351us] +/- 73ms 2024-05-08T08:30:41.314 INFO:teuthology.orchestra.run.smithi149.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 13 +1458us[+1458us] +/- 81ms 2024-05-08T08:30:41.314 INFO:teuthology.orchestra.run.smithi149.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T08:30:41.314 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T08:30:41.324 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T08:30:41.324 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T08:30:41.333 DEBUG:teuthology.orchestra.run.smithi031:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T08:30:41.437 DEBUG:teuthology.orchestra.run.smithi149:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T08:30:41.510 DEBUG:teuthology.orchestra.run.smithi031:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:30:41.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:30:41.541 DEBUG:teuthology.orchestra.run.smithi149:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T08:30:41.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:30:41.571 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T08:30:41.581 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T08:30:41.590 INFO:teuthology.task.internal:Duration was 780.462198 seconds 2024-05-08T08:30:41.590 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T08:30:41.663 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T08:30:41.664 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T08:30:41.667 DEBUG:teuthology.orchestra.run.smithi149:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T08:30:41.706 INFO:teuthology.orchestra.run.smithi031.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:30:41.707 INFO:teuthology.orchestra.run.smithi149.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T08:30:42.062 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T08:30:42.063 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-05-08T08:30:42.063 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T08:30:42.124 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi149.front.sepia.ceph.com 2024-05-08T08:30:42.124 DEBUG:teuthology.orchestra.run.smithi149:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T08:30:42.159 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T08:30:42.159 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T08:30:42.167 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T08:30:42.226 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T08:30:42.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T08:30:42.236 DEBUG:teuthology.orchestra.run.smithi149:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T08:30:42.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T08:30:42.388 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T08:30:42.388 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T08:30:42.413 DEBUG:teuthology.orchestra.run.smithi149:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T08:30:42.439 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T08:30:42.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:30:42.456 DEBUG:teuthology.orchestra.run.smithi149:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:30:42.477 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-05-08T08:30:42.503 INFO:teuthology.orchestra.run.smithi149.stdout:kernel.core_pattern = core 2024-05-08T08:30:42.519 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:30:42.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:30:42.555 DEBUG:teuthology.orchestra.run.smithi149:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T08:30:42.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T08:30:42.573 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T08:30:42.584 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T08:30:42.585 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/remote/smithi031 2024-05-08T08:30:42.585 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T08:30:42.635 DEBUG:teuthology.misc:Transferring archived files from smithi149:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-05-07_21:16:03-rbd-squid-distro-default-smithi/7695986/remote/smithi149 2024-05-08T08:30:42.636 DEBUG:teuthology.orchestra.run.smithi149:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T08:30:42.671 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T08:30:42.672 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T08:30:42.675 DEBUG:teuthology.orchestra.run.smithi149:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T08:30:42.719 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T08:30:42.731 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T08:30:42.731 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T08:30:42.741 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T08:30:42.741 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T08:30:42.743 DEBUG:teuthology.orchestra.run.smithi149:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T08:30:42.756 INFO:teuthology.orchestra.run.smithi031.stdout: 265295 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 08:30 /home/ubuntu/cephtest 2024-05-08T08:30:42.774 INFO:teuthology.orchestra.run.smithi149.stdout: 265809 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 08:30 /home/ubuntu/cephtest 2024-05-08T08:30:42.775 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T08:30:42.786 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T08:30:42.809 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-zstd supported-random-distro$/{centos_latest} thrashers/default thrashosds-health workloads/rbd_fsx_rate_limit} duration: 780.4621982574463 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-05-08T08:30:42.809 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T08:30:42.882 INFO:teuthology.run:pass